Written by Craig Silverman and Rina Tsubaki

With close to 18,000 followers, the Twitter account @ShamiWitness has been a major source of proIslamic State propaganda. In their investigation of the account, British broadcaster Channel 4 reported that ShamiWitness’ tweets “were seen two million times each month, making him perhaps the most influential Islamic State Twitter account.” Channel 4 also reported that two-thirds of Islamic State foreign fighters on Twitter follow the account.

Channel 4 set out to investigate who was behind the account. All it had to go on was the account and its tweets — the person behind ShamiWitness had never shared personal information or anything that might indicate where they were based.

Simon Israel, the Channel 4 correspondent who led the investigation, said in the report that there were no known photos of ShamiWitness.

“But there are moments — and there are always moments — when the hidden trip up,” he said.

Israel said an analysis of the ShamiWitness account revealed that it used to go by a different handle on Twitter: @ElSaltador. At some point, the account owner changed it to @ShamiWitness.

Channel 4 investigators took that previous Twitter handle and searched other social networks to see if they could find anyone using it. That led them to a Google+ account, and then to a Facebook page. There they found photos and other details about a man living in Bangalore who worked as a marketing executive for an Indian company. Soon, they had him on the phone: He confirmed that he was behind the ShamiWitness account.

The result was an investigative story broadcast in December 2014. That report caused the man behind the Twitter account to stop tweeting.

Channel 4 used publicly available data and information to produce journalism that shut down a key source of propaganda and recruitment for the Islamic State.

Journalists, human rights workers and others are constantly making use of open data, user-generated content and other open source information to produce critically important investigations of everything from conflict zones to human rights abuse cases and international corruption.

“Open source information, which is information freely available to anyone through the Internet — think YouTube, Google Maps, Reddit — has made it possible for ANYONE to gather information and source others, through social media networks,” wrote Eliot Higgins on the Kickstarter campaign page for his open source investigations website, Bellingcat. “Think the Syrian Civil War. Think the Arab Spring.”

The abundance of open source information available online and in databases means that just about any investigation today should incorporate the search, gathering and verification of open source information. This has become inseparable from the work of cultivating sources, securing confidential information and other investigative tactics that rely on hidden or less-public information. Journalists and other who develop and maintain the ability to properly search, discover, analyze and verify this material will deliver better, more comprehensive investigations.

Higgins, who also goes by the pseudonym Brown Moses, is living proof of the power of open source information when combined with dedication and strong verification practices. He has become an internationally recognized expert in the Syrian conflict and the downing of Flight MH17 in Ukraine, to name but two examples. His website, Bellingcat, is where he and others now use open source materials to produce unique and credible investigate work.

In February 2015, Bellingcat launched a project to track the vehicles being used in the conflict in Ukraine. They invited the public to submit images or footage of military vehicles spotted in the conflict zone, and to help analyze images and footage that had been discovered from social networks and other sources. In its first week of operation, the project added 71 new entries to the vehicles database, almost doubling the amount of information they had previously collected. These were photos, videos and other pieces of evidence that were gathered from publicly available sources, and they told the story of the conflict in a way no one had before.

It’s all thanks to open source information and user-generated content. As chapters and case studies in this Handbook detail, this same material is being used by investigative journalists in Africa and by groups such as Amnesty International and WITNESS to expose fraud, document war crimes and to help the wrongly accused defend themselves in court.

This companion to the original Verification Handbook offers detailed guidance and illustrative case studies to help journalists, human rights workers and others verify and use open source information and user-generated content in service of investigative projects.

With so much information circulating and available on social networks, in public databases and via other open sources, it’s essential that journalists and others are equipped with the skills and knowledge to search, research and verify this information in order to use it in accurate and ethical ways.

This Handbook provides the fundamentals of online search and research techniques for investigations; details techniques for UGC investigations; offers best practices for evaluating and verifying open data; provides workflow advice for fact-checking investigative projects; and outlines ethical approaches to incorporating UGC in investigations.

The initial Verification Handbook focused on verification fundamentals and offered step-by-step guidance on how to verify user-generated content for breaking news coverage. This companion Handbook goes deeper into search, research, fact-checking and data journalism techniques and tools that can aid investigative projects. At the core of each chapter is a focus on enabling you to surface credible information from publicly available sources, while at the same time offering tips and techniques to help test and verify what you’ve found.

As with the verification of user-generated content in breaking news situations, some fundamentals of verification apply in an investigative context. Some of those fundamentals, which were detailed in the original Handbook, are:

• Develop human sources.
  

• Contact people, and talk to them.
  

• Be skeptical when something looks, sounds or seems too good to be true.
  

• Consult multiple, credible sources.
  

• Familiarize yourself with search and research methods, and new tools.
  

• Communicate and work together with other professionals — verification is a team sport.

Journalist Steve Buttry, who wrote the Verification Fundamentals chapter in the original Handbook, said that verification is a mix of three elements:

• A person’s resourcefulness, persistence, skepticism and skill
  

• Sources’ knowledge, reliability and honesty, and the number, variety and reliability of sources you can find and persuade to talk
  

• Documentation

This Handbook has a particular focus on the third element: documentation. Whether it is using search engines more effectively to gather documentation, examining videos uploaded to YouTube for critical evidence, or evaluating data gathered from an entity or database, it’s essential that investigators have the necessary skills to acquire and verify documentation.

Just as we know that human memory is faulty and that sources lie, we must also remember that documents and data aren’t always what they appear. This Handbook offers some fundamental guidance and case studies to help anyone use open source information and user-generated content in investigations — and to verify that information so that it buttresses an investigation and helps achieve the goal of bringing light to hidden truths.

“In today’s digital environment, where rumors and false contents circulate, journal- ists need to be able to actively sort out true, authentic materials from the fakes. This groundbreaking handbook is a must-read for journalists dealing with all types of user generated contents.”

- Wilfried Ruetten, Director, The European Journalism Centre (EJC)

“Accurate information can be a life-saving resource during a humanitarian crisis, but the circumstances from which these crises emerge are typically the most difficult in which to gather reliable information. This book will help not only journalists but anyone working in humanitarian situations in the field to verify facts on the ground.”

- William Spindler, Spokesman, The United Nations High Commissioner for Refugees (UNHCR)

“This handbook will be essential for journalists covering interreligious and intereth- nic conflicts to report in a more balanced, transparent and accurate way, and ulti- mately help defuse tensions across and within communities.”

- Matthew Hodes, Director, The United Nations Alliance of Civilizations (UNAOC)

“In these times, knowing what is real and how to verify news and other information is essential. This handbook provides essential tools for everyone, journalism and con- sumer.”

- Howard Finberg, Director of Training Partnerships and Alliances, The Poynter Institute

“Getting the facts right is a cardinal principle of journalism but media struggle to be ethical when a big story is breaking. This handbook helps news makers keep faith with truth-telling - even when online speculation is rampant.”

- Aidan White, Director, The Ethical Journalism Network (EJN)

“It’s all about the right information at the right time in the right place. When there is limited access to the disaster-affected areas, it’s crucial for aid workers to gather information via social networks effectively. This handbook would be useful for aid workers working on the ground, as well as online volunteers.”

- Christoph Dennenmoser, Team Lead Urgent Needs, Humanity Road Inc.

Craig Silverman is the media editor of BuzzFeed News, where he leads a global beat covering platforms, online misinformation and media manipulation. He previously edited the “Verification Handbook” and the “Verification Handbook for Investigative Reporting,” and is the author of, "Lies, Damn Lies, and Viral Content: How News Websites Spread (and Debunk) Online Rumors, Unverified Claims and Misinformation."

In December 2019, Twitter user @NickCiarelli shared a video he said showed a dance routine being adopted by supporters of Michael Bloomberg’s presidential campaign. The video’s lackluster enthusiasm and choreography immediately helped it rack up retweets and likes, mostly from people who delighted in mocking it. The video eventually attracted more than 5 million views on Twitter.

Ciarelli’s Twitter bio said he was an intern for the Bloomberg campaign, and his subsequent tweets included proof points such as a screenshot of an email from an alleged Bloomberg campaign staffer approving budget for the video.

But a quick Google search of Ciarelli’s name showed he’s a comedian who has created humor videos in the past. And that email from a Bloomberg staffer? It was sent by Ciarelli’s frequent comedic partner, Brad Evans. That information was also just a Google search away.

But in the first minutes and hours, some believed the cringeworthy video was an official Bloomberg production.

Maggie Haberman, a prominent New York Times political reporter, tweeted that journalists who covered Bloomberg’s previous mayoral campaigns had reason to not dismiss it right away:

Knowledge can take many forms, and in this new digital environment, journalists have to be wary of relying too much on any given source of information — even if it’s their own firsthand experience.

Apparently, some reporters who knew Bloomberg and his style of campaigning felt the video could be real. At the same time, journalists who knew nothing about Bloomberg and chose to judge the video by its source could have found the correct answer immediately — in this case, simply Googling the name of the man who shared it.

The point isn’t that experience covering Bloomberg is bad. It’s that at any given moment we can be led astray by what we think we know. And in some cases our base of knowledge and experience can even be a negative. We can also be fooled by digital signals such as retweets and views, or by efforts to manipulate them.

As the Bloomberg video showed, it takes little effort to create misleading signals like a Twitter bio or a screenshot of an email that appears to back up the content and claim. These in turn help content go viral. And the more retweets and likes it racks up, the more those signals will convince some that the video could be real.

Of course, there are far more devious examples than this one. Unlike Ciarelli, the people behind information operations and disinformation campaigns rarely reveal the ruse. But this case study shows how confusing and frustrating it is for everyone, journalists included, to navigate an information environment filled with easily manipulated signals of quality and trust.

Trust is the foundation of society. It informs and lubricates all transactions and is key to human connection and relationships. But it’s dangerous to operate with default trust in our digital environment.

If your default is to trust that the Twitter accounts retweeting a video are all amplifying it organically, you will get gamed. If you trust that the reviews on a product are all from real customers, you’ll waste your money. If you trust that every news article in your news feed represent an unbiased collection of what you most need to see, you will end up misinformed.

This reality is important for every person to recognize, but it’s essential for journalists. We are being targeted by coordinated and well-funded campaigns to capture our attention, trick us into amplifying messages, and bend us to the will of states and other powerful forces.

The good news is this creates an opportunity — and imperative — for investigation.

This handbook draws on the knowledge and experience of top journalists and researchers to provide guidance on how to execute investigations of digital media manipulation, disinformation and information operations.

We are operating in a complex and rapidly evolving information ecosystem. It requires an equally evolving approach built on testing our assumptions, tracking and anticipating adversaries, and applying the best of open-source investigation and traditional reporting techniques. The vulnerabilities in our digital and data-driven world require journalists to question and scrutinize every aspect of it and apply our skills to help guide the public to accurate, trustworthy information. It also requires journalists to think about how we can unwittingly give oxygen to bad actors and campaigns designed to exploit us, and rush to point fingers at state actors when the evidence does not support it.

The goal of this handbook is to equip journalists with the skills and techniques needed to do this work effectively and responsibly. It also offers basic grounding in the theory, context and mindset that enable journalists to deliver work of high quality that informs the public, exposes bad actors, and helps improve our information environment. But the first thing to understand is that hands-on knowledge and tools are useless unless you approach this work with the right mindset.

This means understanding that everything in the digital environment can be gamed and manipulated, and to recognize the wide variety of people and entities with incentive to do so. The beauty of this environment is that there is often, though not always, a trail of data, interactions, connections and other digital breadcrumbs to follow. And much of it can be publicly available if you know where and how to look.

Investigating the digital means taking nothing at face value. It means understanding that things which appear to be quantifiable and data-driven — likes, shares, retweets, traffic, product reviews, advertising clicks — are easily and often manipulated. It means recognizing that journalists are a key focus of media manipulation and information operations, both in terms of being targeted and attacked, as well as being seen as a key channel to spread mis- and disinformation. And it means equipping yourself and your colleagues with the mindset, techniques and tools necessary to ensure that you’re offering trusted, accurate information — and not amplifying falsehoods, manipulated content or troll campaigns.

At the core of the mindset is the digital investigation paradox: By trusting nothing at first, we can engage in work that reveals what we should and should not trust. And it enables us to produce work that the communities we serve are willing and able to trust.

Along with that, there are some fundamentals that you will see emphasized repeatedly in chapters and case studies:

• Think like an adversary. Each new feature of a platform or digital service can be exploited in some way. It’s critical to put yourself in the shoes of someone looking to manipulate the environment for ideological, political, financial or other reasons. When you look at digital content and messages, you should consider the motivations driving its creation and propagation. It’s also essential to stay abreast of the latest techniques being used by bad actors, digital marketers and others whose livelihood relies on finding new ways to gain attention and earn revenue the digital environment.

• Focus on actors, content, behavior and networks. The goal is to analyze the actors, content and behavior and how they are to document how they might be working in unison as a network. By comparing and contrasting these four things with each other, you can begin to understand what you’re seeing. As you’ll see in multiple chapters and case studies, a fundamental approach is to start with one piece of content or an entity such as a website and pivot on it to identify a larger network through behavior and other connections. This can involve examining the flow of content and actors across platforms, and occasionally into different languages.
  

• Monitor and collect. The best way to identify media manipulation and disinformation is to look for it all the time. Ongoing monitoring and tracking of known actors, topics and communities of interest is essential. Keep and organize what you find, whether in spreadsheets, screenshot folders or by using paid tools like Hunchly.
  

• Be careful with attribution. It’s sometimes impossible to say exactly who’s behind a particular account, piece of content, or a larger information operation. One reason is that actors with different motives can behave in similar ways, and produce or amplify the same kind of content. Even the platforms themselves — which have far better access to data and more resources — make attribution mistakes. The most successful and compelling evidence usually combines digital proof with information from inside sources — an ideal mix of online and traditional investigative work. That’s becoming even more difficult as state actors and others evolve and find new ways to hide their fingerprints. Attribution is difficult; getting it wrong will undermine all of the careful work that led up to it.

Finally, a note on the two handbooks that preceded this edition. This work builds on the foundations of the first edition of the Verification Handbook and the Verification Handbook for Investigative Reporting. Each offers fundamental skills for monitoring social media, verifying images, video and social media accounts, and using search engines to identify people, companies and other entities.

Many of the chapters and case studies in this handbook are written with the assumption that readers possess the basic knowledge laid out in these previous publications, particularly the first handbook. If you are struggling to follow along, I encourage you to start with the first handbook.

Now, let’s get to work.

Data Journalism in Question

What is data journalism? What is it for? What might it do? What opportunities and limitations does it present? Who and what is involved in making and making sense of it? This book is a collaborative experiment responding to these and other questions.

It follows on from another edited book, The Data Journalism Handbook: How Journalists Can Use Data to Improve the News (O’Reilly Media, 2012). Both books assemble a plurality of voices and perspectives to account for the evolving field of data journalism.

The first edition started through a “book sprint” at MozFest in London in 2011, which brought together journalists, technologists, advocacy groups and others in order to write about how data journalism is done. As we wrote in the introduction, it aimed to “document the passion and enthusiasm, the vision and energy of a nascent movement”, to provide “stories behind the stories” and to let “different voices and views shine through”(Gray et al., 2012).

The 2012 edition is now translated into over a dozen languages – including Arabic, Chinese, Czech, French, Georgian, Greek, Italian, Macedonian, Portuguese, Russian, Spanish and Ukrainian – and is used for teaching at many leading universities, as well as teaching and training centres around the world, as well as being a well-cited source for researchers studying the field.

While the 2012 book is still widely used (and this book is intended to complement rather than to replace it), a great deal has happened since 2012.

On the one hand, data journalism has become more established. In 2011 data journalism as such was very much a field “in the making”, with only a handful of people using the term. It has subsequently become socialised and institutionalised through dedicated organisations, training courses, job posts, professional teams, awards, anthologies, journal articles, reports, tools, online communities, hashtags, conferences, networks, meetups, mailing lists and more.

There is also broader awareness of the term through events which are conspicuously data-related, such as the Panama Papers, which whistleblower Edward Snowden then characterised as the “biggest leak in the history of data journalism”(Snowden, 2016).

On the other hand, data journalism has become more contested. The 2013 Snowden leaks helped to establish a transnational surveillance apparatus of states and technology companies as a matter of fact rather than speculation. These leaks suggested how citizens were made knowable through big data practices, showing a darker side to familiar data-making devices, apps and platforms (Gray & Bounegru, 2019).

In the United States the launch of Nate Silver’s dedicated data journalism outlet FiveThirtyEight in 2014 was greeted by a backlash for its overconfidence in particular kinds of quantitative methods and its disdain for “opinion journalism” (Byers, 2014).

While Silver was acclaimed as “lord and god of the algorithm” by The Daily Show’s Jon Stewart for successfully predicting the outcome of the 2012 elections, the statistical methods that he advocated were further critiqued and challenged after the election of Donald Trump in 2016.

These elections along with the Brexit vote in the UK and the rise of populist right-wing leaders around the world, were said to correspond with a “post-truth” moment (Davies, 2016), characterised by a widespread loss of faith in public institutions, expert knowledge and the facts associated with them, and the mediation of public and political life by online platforms which left their users vulnerable to targeting, manipulation and misinformation.1

Whether the so-called “post-truth” moment is taken as evidence of failure or as a call to action, one thing is clear: Data can no longer be taken for granted, and nor can data journalism. Data does not just provide neutral and straightforward representations of the world, but is rather entangled with politics and culture, money and power. Institutions and infrastructures underpinning the production of data – from surveys to statistics, climate science to social media platforms – have been called into question.

At the time of writing, as the COVID-19 pandemic continues to roll on around the world, numbers, graphs and rankings have become widely shared, thematized, politicized and depoliticized—as exemplified by daily circulating epidemiological charts referred to by the now ubiquitous public health strategy to “flatten the curve.”

At the same time, the fragility and provisionality of such data has been widely reported on, with concerns around the under-reporting, non-reporting and classification of cases, as well as growing awareness of the societal and political implications of different kinds of data from sources—from hospital figures to research estimates to self-reporting to transactional data from tracing apps. The pandemic has broadened awareness of not just using but also critically reporting on numbers and data.

Thus one might ask of the use of data in journalism:

-Which data, whose data and by which means?

-Data about which issues and to what end?

-Which kinds of issues are data-rich and which are data-poor, and why?

-Who has the capacity to benefit from data and who doesn’t?

-What kinds of publics does data assemble, which kinds of capacities does it support, what kinds of politics does it enact and what kinds of participation does it engender?

Towards a Critical Data Practice

Rather than bracketing such questions and concerns, this book aims to “stay with the trouble” as the prominent feminist scholar Donna Haraway (2016) puts it.2

Instead of treating the relevance and importance of data journalism as an assertion, we treat this as a question which can be addressed in multiple ways. The collection of chapters gathered in the book aim to provide a richer story about what data journalism does, with and for whom.

Through our editorial work we have encouraged both reflection and a kind of modesty in articulating what data journalism projects can do, and the conditions under which they can succeed.

This entails the cultivation of a different kind of precision in accounting for data journalism practice: specifying the situations in which it develops and operates. Such precision requires broadening the scope of the book to include not just the ways in which data is analysed, created and used in the context of journalism but also more about the social, cultural, political and economic circumstances in which such practices are embedded.

The subtitle of this new book is “towards a critical data practice”, and reflects both our aspiration as editors to bring critical reflection to bear on data journalism practices, as well as reflecting the increasingly critical stances of data journalism practitioners.

The notion of “critical data practice” is a nod to Philip E. Agre’s notion of “critical technical practice”, which he describes in terms of having “one foot planted in the craft work of design and the other foot planted in the reflexive work of critique” (Agre, 1997, p. 155).

As we have written about elsewhere, our interest in this book is understanding how critical engagements with data might modify data practices, making space for public imagination and interventions around data politics (Gray, 2018; Gray et al., 2018).

Alongside contributions from data journalists and practitioners writing about what they do, the book also includes chapters from researchers whose work may advance critical reflection on data journalism practices, from fields such as anthropology, science and technology studies, (new) media studies, internet studies, platform studies, the sociology of quantification, journalism studies, indigenous studies, feminist studies, digital methods and digital sociology.

Rather than assuming a more traditional division of labour such that researchers provide critical reflection and practitioners offer more instrumental tips and advice, we have sought to encourage researchers to consider the practical salience of their work, and to provide practitioners with space to reflect on what they do outside of their day-to-day deadlines.

None of these different perspectives exhaust the field, and our objective is to encourage readers to attend to the different aspects of how data journalism is done. In other words, this book is intended to function as an multidisciplinary conversation starter, and – we hope – a catalyst for collaborations.

We do not assume that “data journalism” refers to a unified set of practices. Rather it is a prominent label which refers to a diverse set of practices which can be empirically studied, specified and experimented with. As one recent review puts it, we need to interrogate the “how of quantification as much as the mere fact of it”, the effects of which “depend on intentions and implementation” (Berman & Hirschman, 2018).

Our purpose is not to stabilise how data journalism is done, but rather to draw attention to its manifold aspects and open up space for doing it differently.

A Collective Experiment

It is worth briefly noting what this book is not. It is not just a textbook or handbook in the conventional sense: the chapters don’t add up to an established body of knowledge, but are rather intended to indicate interesting directions for further inquiry and experimentation.

The book is not just a practical guidebook of tutorials or “how tos”: There are already countless readily available materials and courses on different aspects of data practice (e.g. data analysis and data visualisation). It is not just a book of “behind the scenes” case studies: There are plenty of articles and blog posts showing how projects were done, including interviews with their creators.

It is not just a book of recent academic perspectives: there is an emerging body of literature on data journalism scattered across numerous books and journals.3

Rather, the book has been designed as a collective experiment in accounting for data journalism practices and a collective invitation to explore how such practices may be modified.

It is collective in that, as with the first edition, we have been able to assemble a comparatively large number of contributors (over 70) for a short book. The editorial process has benefitted from recommendations from contributors during email exchanges.

A workshop with a number of contributors at the International Journalism Festival in Perugia in 2018 provided an opportunity for exchanges and reflection. A “beta” version of the book has been released online to provide an opportunity to publicly preview a selection of chapters before the printed version of the book is published and to elicit comments and encounters before the book takes its final shape.

Through what could be considered a kind of curated “snowball editorial”, we have sought to follow how data journalism is done by different actors, in different places, around different topics, through different means. Through the process we have trawled through many shortlists, longlists, outlets and datasets to curate different perspectives on data journalism practices. Although there were many, many more contributors we would have liked to include, we had to operate within the constraints of a printable book, as well as giving voice to a diversity of geographies, themes, concerns and genders.

It is experimental in that the chapters provide different perspectives and provocations on data journalism, which we invite readers to further explore through actively configuring their own blends of tools, data sets, methods, texts, publics and issues. Rather than inheriting the ways of seeing and ways of knowing that have been “baked into” elements such as official data sets or social media data, we encourage readers to enrol them into the service of their own lines of inquiry.

This follows the spirit of “critical analytics” and “inventive methods” which aim to modify the questions which are asked and the way problems are framed (Lury & Wakeford, 2012; Rogers, 2018).

Data journalism can be viewed not just in terms of how things are represented, but in terms of how it organises relations – such that it is not just a matter of producing data stories (through collecting, analysing, visualising and narrating data), but also attending to who and what these stories bring together (including audiences, sources, methods, institutions and social media platforms). Thus we may ask, as Noortje Marres recently put it: “What are the methods, materials, techniques and arrangements that we curate in order to create spaces where problems can be addressed differently?”4.

The chapters in this book show how data journalism can be an inventive, imaginative, collaborative craft, highlighting how data journalists interrogate official data sources, make and compile their own data, try new visual and interactive formats, reflect on the effects of their work and make their methods accountable and code re-usable.

If the future of data journalism is uncertain, then we hope that readers of this book will join us in both critically taking stock of what journalism is and has been, as well as intervening to shape its future. As with all works, the success, failure and ultimate fate of this book-as-experiment ultimately lies with you, its readers, what you do with it, what it prompts and the responses it elicits.

The cover image of this book is a photograph of Sarah Sze’s Fixed Points Finding a Home in the modern art museum Mudam Luxembourg, for which we are most grateful to the artist, her gallery and the museum for their permission to reproduce.5

While it might not seem an obvious choice to put a work of sculpture on the cover of a book about journalism, we thought this image might encourage a relational perspective on data journalism as a kind of curatorial craft, assembling and working with diverse materials, communities and infrastructures to generate different ways of knowing, narrating and seeing the world at different scales and temporalities.

Rather than focusing on the outputs of data journalism (e.g., with screenshots of visualizations or interactives), we wanted to reflect the different kinds of processes and collectives involved in doing journalism with data. Having both serendipitously encountered and been deeply absorbed by Sze’s exhibitions at the Mudam, Venice Biennale, ZKM, the Tate and beyond, we thought her work could provide a different (and hopefully less familiar) vantage point on the practice of data journalism which would resonate with relational perspectives on information infrastructures and “data assemblages.”6

Her installations embody a precise and playful sensibility towards repurposing found materials that visually paralleled what we were hoping to emphasize with our editorial of different accounts of data journalism for the book. Bruno Latour recently wrote that Sze’s approach to assembling materials can be considered to affirm “compositional discontinuities” (Latour, 2020) —which sits well with our hopes to encourage “critical data practice” and to tell stories both with and about the diverse materials and actors involved in data journalism, as we discuss further below, as well as with our editorial approach in supporting the different styles, voices, vernaculars and interests of the chapters in this book.

An Overview of the Book

To stay true to our editorial emphasis on specifying the setting, we note that the orientation of the book and its selection of chapters is coloured by our interests and those of our friends, colleagues and networks at this particular moment—including growing concerns about climate change, environmental destruction, air pollution, tax avoidance, (neo)colonialism, racism, sexism, inequality, extractivism, authoritarianism, algorithmic injustice and platform labour.

The chapters explore how data journalism makes such issues intelligible and experienceable, as well as the kinds of responses it can mobilize. The selection of chapters also reflects our own oscillations between academic research, journalism and advocacy, as well as the different styles of writing and data practice associated with each of these.

We remain convinced of the generative potential of encounters between colleagues in these different fields, and several of the chapters attest to successful cross-field collaborations. As well as exploring synergies and commonalities, it is also worth noting at the outset (as astute readers will notice) that there are differences, tensions and frictions between the perspectives presented in the various chapters, including different histories and origin stories; different views on methods, data and emerging technologies; different views on the desirability of conventionalization and experimentation with different approaches; and different perspectives on what data journalism is, what it is for, its conditions and constraints, how it is organized and the possibilities it presents.

After this introduction, the book starts with a “taster menu” on doing issues with data. This includes a variety of different formats for making sense of different themes in different places—including tracing connections between agricultural commodities, crime, corruption and colonialism across several countries (Sánchez and Villagrán), mapping segregation in the United States (Williams), multiplying memories of trees in Bogotá (Magaña), looking at the people and scenes behind the numbers for home demolitions in occupied East Jerusalem (Haddad), mobilizing for road safety in the Philippines (Rey) and tracking worker deaths in Turkey (Dağ). The chapters in this section illustrate a breadth of practices from visualization techniques to building campaigns to repurposing official data with different analytical priorities.

The second section focuses on how journalists assemble data—an important emerging area which we have sought to foreground in the book and associated research (Gray et al., 2018; Gray & Bounegru, 2019). This includes exploring the making of projects on themes such as knife crime (Barr) and land conflicts (Shrivastava and Paliwal) as well as accounts of how to obtain and work with data in countries where it may be less easy to come by, such as in Cuba (Reyes, Almeida and Guerra) and China (Ma). Assembling data may also be a way of engaging with readers (Coelho) and assembling interested actors around an issue, which may in itself constitute an important outcome of a project. Gathering data may involve the modification of other forms of knowledge production, such as polls and surveys, to the context of journalism (Boros). A chapter on Indigenous data sovereignty (Kukutai and Walter) explores social, cultural and political issues around official data and how to bring other marginalized perspectives to bear on the organization of collective life with data. As well as using numbers as material for telling stories, data journalists may also tell stories about how numbers are made (Verran).

The third section is concerned with different ways of working with data. This includes with algorithms (Stray), code (Simon) and machines (Borges-Rey). Contributors examine emerging issues and opportunities arising from working with sources such as text data (Maseda). Others look at practices for making data journalistic work transparent, accountable and reproducible (Leon; Mazotte). Databases may also afford opportunities for collaborative work on large investigative projects (Díaz-Struck, Gallego and Romera). Feminist thought and practice may also inspire different ways of working with data (D’Ignazio).

The fourth section is dedicated to examining different ways in which data can be experienced, starting with a look at the different formats that data journalism can take (Cohen). Several pieces reflect on contemporary visualization practices (Aisch and Rost), as well as how readers respond to and participate in making sense with visualizations (Kennedy et al.). Other pieces look at how data is mediated and presented to readers through databases (Rahman and Wehrmeyer), web-based interactives (Bentley), TV and radio (de Jong), comics (Luna), and sketching with data (Chalabi and Gray).

The fifth section is dedicated to emerging approaches for investigating data, platforms and algorithms.

Recent journalism projects take the digital as not only offering new techniques and opportunities for journalists, but also new objects for investigation. Examples of this are Bellingcat and BuzzFeed News’ widely shared work on viral content, misinformation and digital culture. 7
Chapters in this section examine different ways of reporting on algorithms (Diakopoulous), as well as how to conduct longer-term collaborations in this area (Elmer). Other chapters look at how to work with social media data to explore how platforms participate in shaping debate, including storytelling approaches (Vo) as well as affinities between digital methods research and data journalism, including how “born digital” data can be used for investigations into web tracking infrastructures (Rogers) as well as about apps and their associated platforms (Weltevrede).

The sixth section is on organizing data journalism, and attends to different types of work in the field which are considered indispensable but not always prominently recognized. This includes how data journalism has changed over the past decade (Rogers); how platforms and the gig economy shape cross-border investigative networks (Cândea); entanglements between data journalism and movements for open data and civic tech (Baack); open-source coding practices (Pitts and Muscato); audience-measurement practices (Petre); archiving data journalism (Broussard); and the role of the #ddj hashtag in connecting data journalism communities on Twitter (Au and Smith).

The seventh section focuses on learning about data journalism as a collaborative process, including data journalism training programmes and the development of data journalism around the world. This includes chapters on teaching data journalism at universities in the United States (Phillips); empowering marginalized communities to tell their stories (Constantaras; Vaca); caution against “digital universalism” and underestimating innova- tion in the “periphery” (Chan); and different approaches for collaborations between journalists and researchers (Radcliffe and Lewis).

Data journalism does not happen in a vacuum. The eighth and final section focuses on situating this practice in relation to its various social, political, cultural and economic settings. A chapter on the genealogies of data journalism in the United States serves to encourage reflection on the various historical practices and ideas which shape it (Anderson). Other chapters look at how data journalism projects are valued through awards (Loosen); different approaches to measuring the impact of data journalism projects (Bradshaw; Green-Barber); issues around data journalism and colonialism (Young and Callison); whether data journalism can live up to its earlier aspirations to become a field of inspired experimentation, interactivity and play (Usher); and data journalism and digital liberalism (Boyer).

Twelve Challenges for Critical Data Practice

Drawing on the time that we have spent exploring data journalism practices through the development of this book, we would like to conclude this introduction to the book with twelve challenges for “critical data practice.”

These consider data journalism in terms of its capacities to shape relations between different actors as well as to produce representations about the world. Having been tested in the context of our “engaged research-led teaching” collaborations at King’s College London and the Public Data Lab,8 they are intended as a prompt for aspiring data journalists, student group projects and investigations, researcher–journalist collaborations, and other activities which aspire to organize collective inquiry with data without taking for granted the infrastructures, environments and practices through which it is produced.

1. How can data journalism projects tell stories both with and about data including the various actors, processes, institutions, infrastructures and forms of knowledge through which data is made?
   
2. How can data journalism projects tell stories about big issues at scale (e.g., climate change, inequality, multinational taxation, migration) while also affirming the provisionality and acknowledging the models, assumptions and uncertainty involved in the production of numbers?
   
3. How can data journalism projects account for the collective character of digital data, platforms, algorithms and online devices, including the interplay between digital technologies and digital cultures?
   
4. How can data journalism projects cultivate their own ways of making things intelligible, meaningful and relatable through data, without simply uncritically advancing the ways of knowing “baked into” data from dominant institutions, infrastructures and practices?
   
5. How can data journalism projects acknowledge and experiment with the visual cultures and aesthetics that they draw on, including through combinations of data visualizations and other visual materials?
   
6. How can data journalism projects make space for public participation and intervention in interrogating established data sources and re-imagining which issues are accounted for through data, and how?
   
7. How might data journalists cultivate and consciously affirm their own styles of working with data, which may draw on, yet remain distinct from, areas such as statistics, data science and social media analytics?
   
8. How can the field of data journalism develop memory practices to archive and preserve their work, as well as situating it in relation to practices and cultures that they draw on?
   
9. How can data journalism projects collaborate around transnational issues in ways which avoid the logic of the platform and the colony, and affirm innovations at the periphery?
10. How can data journalism support marginalized communities to use data to tell their own stories on their own terms, rather than telling their stories for them?
    
11. How can data journalism projects develop their own alternative and inventive ways of accounting for their value and impact in the world, beyond social media metrics and impact methodologies established in other fields?
12. How might data journalism develop a style of objectivity which affirms, rather than minimizes, its own role in intervening in the world and in shaping relations between different actors in collective life?

Words of Thanks

We are most grateful to Amsterdam University Press (AUP), and in particular to Maryse Elliott, for being so supportive with this experimental project, including the publication of an online beta as well as their support for an open access digital version of the book. AUP is perhaps also an apt choice, given that several of the contributors gathered at an early international conference on data journalism which took place in Amsterdam in 2010. Open access funding is supported by a grant from the Netherlands Organisation for Scientific Research (NWO, 324-98-014), thanks to Richard Rogers at the University of Amsterdam.

The vision for the book was germinated through discussions with friends and colleagues associated with the Public Data Lab. We particularly benefited from conversations about different aspects of this book with Andreas Birkbak, Erik Borra, Noortje Marres, Richard Rogers, Tommaso Venturini, Esther Weltevrede, Michele Mauri, Gabriele Colombo and Angeles Briones.

We were also provided with space to develop the direction of this book through events and visits to Columbia University (in discussion with Bruno Latour); Utrecht University; the University of California, Berkeley; Stanford University; the University of Amsterdam; the University of Miami; Aalborg University Copenhagen; Sciences Po, Paris; the University of Cambridge; London School of Economics; Cardiff University; Lancaster University; and the International Journalism Festival in Perugia.

Graduate students taking the MA course in data journalism at King’s College London helped us to test the notion of “critical data practice” which lies at the heart of this book. Our longstanding hope to do another edition was both nurtured and materialized thanks to Rina Tsubaki, who helped to facilitate support from the European Journalism Centre and the Google News Lab. We are grateful to Adam Thomas, Bianca Lemmens, Biba Klomp, Letizia Gambini, Arne Grauls and Simon Rogers for providing us with both editorial independence and enduring support to scale up our efforts.

The editorial assistance of Daniela Demarchi and Oana Bounegru has been tremendously valuable in helping us to chart a clear course through sprawling currents of texts, footnotes, references, emails, shared documents, version histories, spreadsheets and other materials.

Most of all, we would like to thank all of the data journalism practitioners and researchers who were involved in the project (whether through writing, correspondence or discussion) for accompanying us, and for supporting this experiment with their contributions of time, energy, materials and ideas without which the project would not have been possible. This book is, and continues to be, a collective undertaking.

Footnotes

1. For a critical perspective on this term, see Jasanoff, S., & Simmet, H. R. (2017). No funeral bells: Public reason in a “post-truth” age. Social Studies of Science, 47(5), 751–770.

2. Alluding to this work, Verran’s chapter in this book explores how data journalists might stay with the trouble of value and numbers.

3. www.zotero.org/groups/data_journalism_research

4. A question that Noortje Marres asked in her plenary contribution to EASST 2018 in Lancaster: twitter.com/jwyg/status/1023200997668204544

5. Sarah Sze,Fixed Points Finding a Home, 2012 (details). Mixed media. Dimensions variable. Mudam Luxembourg Commission and Collection. Donation 2012—Les Amis des Musées d’Art et d’Histoire Luxembourg. © Artwork: Sarah Sze. Courtesy the artist and Victoria Miro. © Photo: Andrés Lejona/Mudam Luxembourg.

6. For relational perspectives on data infrastructures see, for example, the seminal work of Susan Leigh Star: Star, S. L., & Ruhleder, K. (1996). Steps toward an ecology of infrastructure: Design and access for large information spaces.Information Systems Research, 7, 111–134; Star, S. L. (1999). The ethnography of infrastructure. American Behavioral Scientist, 43, 377–391. For more recent work on “data assemblages,” see, for example: Kitchin, R. (2014). The data Big data, open data, data infrastructures and their consequences. SAGE; Kitchin, R., & Lauriault,T. (2018). Towards critical data studies: Charting and unpacking data assemblages and their work. In J. Thatcher, A. Shears, & J. Eckert (Eds.), Thinking big data in geography: New regimes, new research (pp. 3–20). University of Nebraska Press.

7. www.buzzfeednews.com/topic/fake-news, www.bellingcat.com

8. www.kcl.ac.uk/research/engaged-research-led-teaching and publicdatalab.org

Works Cited

Agre, P. E. (1997). Toward a critical technical practice: Lessons learned in trying to reform AI. In G. Bowker, S. L. Star, B. Turner, & L. Gasser (Eds.), Social science, techni- cal systems, and cooperative work: Beyond the great divide (pp. 130–157). Erlbaum.

Berman, E. P., & Hirschman, D. (2018). The sociology of quantif ication: Where are we now? Contemporary Sociology, 47(3), 257–266. doi.org/10.1177/0094306118767649

Byers, D. (2014, March 19). Knives out for Nate Silver. Politico. www.politico.com/blogs/media/2014/03/knives-out-for-nate-silver-185394.html

Davies, W. (2016, August 24). The age of post-truth politics. The New York Times. www.nytimes.com/2016/08/24/opinion/campaign-stops/the-age-of-post-truth-politics.html

Gray, J. (2018). Three aspects of data worlds. Krisis: Journal for Contemporary Philosophy, 1.

Gray, J., & Bounegru, L. (2019). What a difference a dataset makes? Data journalism and/as data activism. In J. Evans, S. Ruane, & H. Southall (Eds.), Data in society: Challenging statistics in an age of globalisation (pp. 365-374). The Policy Press. doi.org/10.5281/zenodo.1415450

Gray, J., Chambers, L., & Bounegru, L. (Eds.). (2012). The data journalism handbook: How journalists can use data to improve the news. O’Reilly Media.

Gray, J., Gerlitz, C., & Bounegru, L. (2018). Data infrastructure literacy. Big Data & Society, 5(2), 1–13. doi.org/10.1177/2053951718786316

Haraway, D. J. (2016). Staying with the trouble: Making kin in the Chthulucene. Duke University Press. doi.org/10.1215/9780822373780

Latour, B. (2020) Sarah Sze as the sculptor of Critical Zones. In B. Latour & P. Weibel (Eds.), Critical zones: The science and politics of landing on earth (pp. 158–159). The MIT Press.

Lury, C., & Wakeford, N. (Eds.). (2012). Inventive methods: The happening of the social. Routledge.

Rogers, R. (2018). Otherwise engaged: Social media from vanity metrics to critical analytics. International Journal of Communication, 12, 450–472. dare.uva.nl/search?identifier=e7a7c11b-b199-4d7c-a9cb-fdf1dd74d493

Snowden, E. (2016, April 3). Biggest leak in the history of data journalism just went live, and it’s about corruption. Twitter. twitter.com/Snowden/status/716683740903247873

The Data Journalism Handbook was born at a 48 hour workshop at MozFest 2011 in London. It subsequently spilled over into an international, collaborative effort involving dozens of data journalism’s leading advocates and best practitioners.

In the 6 months that passed between the book’s inception to its first full release, hundreds of people have contributed in various ways. While we have done our best to keep track of them all, we have had our fair share of anonymous, pseudonymous and untraceable edits.

To all of those people who have contributed and are not listed below, we say two things. Firstly, thank you. Secondly, can please tell us who you are so that we can give credit where credit is due.

Written by Henk van Ess

Online research is often a challenge for traditional investigative reporters, journalism lecturers and students. Information from the web can be fake, biased, incomplete or all of the above.

Offline, too, there is no happy hunting ground with unbiased people or completely honest governments. In the end, it all boils down to asking the right questions, digital or not. This chapter gives you some strategic advice and tools for digitizing three of the biggest questions in journalism: who, where and when?

1. Who?

Let’s do a background profile with Google on Ben van Beurden, CEO of the Shell Oil Co.

a. Find facts and opinions

The simple two-letter word “is” reveals opinions and facts about your subject. To avoid clutter, include the company name of the person or any other detail you know, and tell Google that both words should be not that far from each other.

The AROUND() operator MUST BE IN CAPITALS. It sets the maximum distance in words between the two terms.

b. What do others say?

This search is asking Google to “Show me PDF documents with the name of the CEO of Shell in it, but exclude documents from Shell.” This will find documents about your subject, but not from the company of the subject itself. This helps you to see what opponents, competitors or opinionated people say about your subject. If you are a perfectionist, go for

inurl:pdf “ben van beurden” –site:shell.*

because you will find also PDFs that are not visible with filetype.

c. Official databases

Search for worldwide official documents about this person. It searches for gov.uk (United Kingdom) but also .gov.au (Australia), .gov.cn (China), .gov (U.S.) and other governmental websites in the world. If you don’t have a .gov website in your country, use the local word for it with the site: operator. Examples would be site:bund.de (Germany) or site:overheid.nl (The Netherlands).

With this query, we found van Beurden’s planning permission for his house in London, which helped us to find his full address and other details.

d. United Nations

You are now searching in any United Nations-related organization. In this example, we find the Shell CEO popping up in a paper about “Strategic Approach to International Chemicals Management.” And we found his full name, the name of his wife, and his passport number at the time when we did this search. Amazing.

e. Find the variations

With this formula you can find result that use different spellings of the name. You will receive documents with the word Shell, but not those that include “Ben” as the first name. With this, you will find out that he is also referred to as Bernardus van Beurden. (You don’t need to enter a dot [.] because Google will ignore points.) Now repeat steps a, b, c and d with this new name.

2. Where

a. Use photo search in Topsy

You can use http://topsy.thisisthebrigade.com to find out where your subject was, by analyzing his mentions (1) over time (2) and by looking at the photos (3) that others posted on Twitter. If you’d rather research a specific period, go for “Specific Range” in the time menu.

b. Use Echosec

With Echosec, you can search social media for free. In this example, I entered the address of Shell HQ (1) in hopes of finding recent (2) postings from people who work there (3).

c. Use photo search in Google Images

Combine all you know about your subject in one mighty phrase. In the below example, I’m searching for a jihadist called @MuhajiriShaam (1) but not the account @MuhajiriShaam01 (2) on Twitter (3). I just want to see the photos he posted on Twitter between Sept. 25 and Sept. 29, 2014 (4).

3. When

a. Date search

Most of the research you do is not based on today, but an earlier period. Always tell your search engine this. Go back in time.

Let’s investigate a fire in a Dutch chemical plant called Chemie-Pack. The fire happened on Jan. 5, 2011. Perhaps you want to investigate if dangerous chemicals were stored at the plant. Go to images.google.com, type in Chemie-pack (1) and just search before January 2011 (2). The results offer hundreds of photos from a youth fire department that visited the company days before the fire. In some photos, you can see barrels with names of chemicals on them. We used this to establish which chemicals were stored in the plant days before the fire.

b. Find old data with archive.org

Websites often cease to exist. There is a chance you can still view them by using archive.org. This tool can do its work only if you know the URL of the webpage you want to see. The problem is that often the link is gone and therefore you don’t know it. So how do you find a seemingly disappeared URL?

Let’s assume we want to find the home page of a dead actress called Lana Clarkson.

Step One: Find an index

Find a source about the missing page. In this case, we can use her Wikipedia page.

Step Two: Put the index in the time machine

Go to archive.org and enter the URL of her Wikipedia page, http://en.wikipedia.org/wiki/Lana_Clarkson. Choose the oldest available version, March 10, 2004. There it says the home page was http://www.lanaclarkson.com.

Step Three: Find the original website

Now type the link in archive.org, but add a backslash and an asterisk to the URL: https://web.archive.org/web/*/http://www.lanaclarkson.com/*

All filed links are now visible. Unfortunately, in this case, you won’t find that much. Clarkson became famous only after her death. She was shot and killed by famed music producer Phil Spector in February 2003.

The following people have drafted or otherwise directly contributed to text which is in the current version of the book. The illustrations are by graphic designer Kate Hudson.

Written by Paul Myers

Search engines are an intrinsic part of the array of commonly used “open source” research tools. Together with social media, domain name look-ups and more traditional solutions such as newspapers and telephone directories, effective web searching will help you find vital information to support your investigation.

Many people find that search engines often bring up disappointing results from dubious sources. A few tricks, however, can ensure that you corner the pages you are looking for, from sites you can trust. The same goes for searching social networks and other sources to locate people: A bit of strategy and an understanding of how to extract what you need will improve results.

This chapter focuses on three areas of online investigation:

1. Effective web searching.
2. Finding people online.
3. Identifying domain ownership.

1. Effective web searching

Search engines like Google don’t actually know what web pages are about. They do, however, know the words that are on the pages. So to get a search engine to behave itself, you need to work out which words are on your target pages.

First off, choose your search terms wisely. Each word you add to the search focuses the results by eliminating results that don’t include your chosen keywords.

Some words are on every page you are after. Other words might or might not be on the target page. Try to avoid those subjective keywords, as they can eliminate useful pages from the results.

Use advanced search syntax.

Most search engines have useful so-called hidden features that are essential to helping focus your search and improve results.

Optional keywords

If you don’t have definite keywords, you can still build in other possible keywords without damaging the results. For example, pages discussing heroin use in Texas might not include the word “Texas”; they may just mention the names of different cities. You can build these into your search as optional keywords by separating them with the word OR (in capital letters).

You can use the same technique to search for different spellings of the name of an individual, company or organization.

Search by domain

You can focus your search on a particular site by using the search syntax “site:” followed by the domain name.

For example, to restrict your search to results from Twitter:

To add Facebook to the search, simply use “OR” again:

You can use this technique to focus on a particular company’s website, for example. Google will then return results only from that site.

You can also use it to focus your search on municipal and academic sources, too. This is particularly effective when researching countries that use unique domain types for government and university sites

Note: When searching academic websites, be sure to check whether the page you find is written or maintained by the university, one of its professors or one of the students. As always, the specific source matters.

Searching for file types

Some information comes in certain types of file formats. For instance, statistics, figures and data often appear in Excel spreadsheets. Professionally produced reports can often be found in PDF documents. You can specify a format in your search by using “filetype:” followed by the desired data file extension (xls for spreadsheet, docx for Word documents, etc.).

2. Finding people

Groups can be easy to find online, but it’s often trickier to find an individual person. Start by building a dossier on the person you’re trying to locate or learn more about. This can include the following:

• The person’s name, bearing in mind:
  • Different variations (does James call himself “James,” “Jim,” “Jimmy” or “Jamie”?).
  • The spelling of foreign names in Roman letters (is Yusef spelled “Yousef” or “Yusuf”?).
  • Did the names change when a person married?
  • Do you know a middle name or initial?

• The town the person lives in and or was born in.

• The person’s job and company.

• Their friends and family members’ names, as these may appear in friends and follower lists.

• The person’s phone number, which is now searchable in Facebook and may appear on web pages found in Google searches.

• Any of the person’s usernames, as these are often constant across various social networks.

• The person’s email address, as these may be entered into Facebook to reveal linked accounts. If you don’t know an email address, but have an idea of the domain the person uses, sites such as email-format can help you guess it.

• A photograph, as this can help you find the right person, if the name is common.

Advanced social media searches: Facebook

Facebook’s newly launched search tool is amazing. Unlike previous Facebook searches, it will let you find people by different criteria including, for the first time, the pages someone has Liked. It also enables you to perform keyword searches on Facebook pages.

This keyword search, the most recent feature, sadly does not incorporate any advanced search filters (yet). It also seems to restrict its search to posts from your social circle, their favorite pages and from some high-profile accounts.

Aside from keywords in posts, the search can be directed at people, pages, photos, events, places, groups and apps. The search results for each are available in clickable tabs.

For example, a simple search for Chelsea will find bring up related pages and posts in the Posts tab:

The People tab brings up people named Chelsea. As with the other tabs, the order of results is weighted in favor of connections to your friends and favorite pages.

The Photos tab will bring up photos posted publicly, or posted by friends that are related to the word Chelsea (such as Chelsea Clinton, Chelsea Football Club or your friends on a night out in the Chelsea district of London).

The real investigative value of Facebook’s search becomes apparent when you start focusing a search on what you really want.

For example, if you are investigating links between extremist groups and football, you might want to search for people who like The English Defence League and Chelsea Football Club. To reveal the results, remember to click on the “People” tab.

This search tool is new and Facebook are still ironing out the creases, so you may need a few attempts at wording your search. That said, it is worth your patience.

Facebook also allows you to add all sorts of modifiers and filters to your search. For example, you can specify marital status, sexuality, religion, political views, pages people like, groups they have joined and areas they live or grew up in. You can specify where they studied, what job they do and which company they work for. You can even find the comments that someone has added to uploaded photos. You can find someone by name or find photos someone has been tagged in. You can list people who have participated in events and visited named locations. Moreover, you can combine all these factors into elaborate, imaginative, sophisticated searches and find results you never knew possible. That said, you may find still better results searching the site via search engines like Google (add “site:facebook.com” to the search box).

Advanced social media searches: Twitter

Many of the other social networks allow advanced searches that often go far beyond the simple “keyword on page” search offered by sites such as Google. Twitter’s advanced search, for example, allows you to trace conversations between users and add a date range to your search.

Twitter allows third-party sites to use its data and create their own exciting searches.
Followerwonk, for example, lets you search Twitter bios and compare different users. Topsy has a great archive of tweets, along with other unique functionality.

Advanced social media searches: LinkedIn

LinkedIn will let you search various fields including location, university attended, current company, past company or seniority.

You have to log in to LinkedIn in order to use the advanced search, so remember to check your privacy settings. You wouldn’t want to leave traceable footprints on the profile of someone you are investigating!

You can get into LinkedIn’s advanced search by clicking on the link next to the search box. Be sure, also, to select “3rd + Everyone Else” under relationship. Otherwise , your search will include your friends and colleagues and their friends.

LinkedIn was primarily designed for business networking. Its advanced search seems to have been designed primarily for recruiters, but it is still very useful for investigators and journalists. Personal data exists in clearly defined subject fields, so it is easy to specify each element of your search.

You can enter normal keywords, first and last names, locations, current and previous employers, universities and other factors. Subscribers to their premium service can specify company size and job role.

LinkedIn will let you search various fields including location, university attended, current company, past company and seniority.

Other options

Sites like Geofeedia and Echosec allow you to find tweets, Facebook posts, YouTube videos, Flickr and Instagram photos that were sent from defined locations. Draw a box over a region or a building and reveal the social media activity. Geosocialfootprint.com will plot a Twitter user’s activity onto a map (all assuming the users have enabled location for their accounts).

Additionally, specialist “people research” tools like Pipl and Spokeo can do a lot of the hard legwork for your investigation by searching for the subject on multiple databases, social networks and even dating websites. Just enter a name, email address or username and let the search do the rest. Another option is to use the multisearch tool from Storyful. It’s a browser plugin for Chrome that enables you to enter a single search term, such as a username, and get results from Twitter, Instagram, YouTube, Tumblr and Spokeo. Each site opens in a new browser tab with the relevant results.

Searching by profile pic

People often use the same photo as a profile picture for different social networks. This being the case, a reverse image search on sites like TinEye and Google Images, will help you identify linked accounts.

3. Identifying domain ownership

Many journalists have been fooled by malicious websites. Since it’s easy for anyone to buy an unclaimed .com, .net or .org site, we should not go on face value. A site that looks well produced and has authentic-sounding domain name may still be a political hoax, false company or satirical prank.

Some degree of quality control can be achieved by examining the domain name itself. Google it and see what other people are saying about the site. A “whois” search is also essential. DomainTools.com is one of many sites that offers the ability to perform a whois search. It will bring up the registration details given by the site owner the domain name was purchased.

For example, the World Trade Organization was preceded by the General Agreement on Tariffs and Trades (GATT). There are, apparently, two sites representing the WTO. There’s wto.org (genuine) and gatt.org (a hoax). A mere look at the site hosted at gatt.org should tell most researchers that something is wrong, but journalists have been fooled before.

A whois search dispels any doubt by revealing the domain name registration information. Wto.org is registered to the International Computing Centre of the United Nations. Gatt.org, however, is registered to “Andy Bichlbaum” from the notorious pranksters the Yes Men.

Whois is not a panacea for verification. People can often get away with lying on a domain registration form. Some people will use an anonymizing service like Domains by Proxy, but combining a whois search with other domain name and IP address tools forms a valuable weapon in the battle to provide useful material from authentic sources

This book is intended to be a useful resource for anyone who thinks that they might be interested in becoming a data journalist, or dabbling in data journalism.

Lots of people have contributed to writing it, and through our editorial we have tried to let their different voices and views shine through. We hope that it reads like a rich and informative conversation about what data journalism is, why it is important, and how to do it.

Lamentably the act of reading this book will not supply you with a comprehensive repertoire of all if the knowledge and skills you need to become a data journalist. This would require a vast library manned by hundreds of experts able to help answer questions on hundreds of topics. Luckily this library exists and it is called the internet. Instead, we hope this book will give you a sense of how to get started and where to look if you want to go further. Examples and tutorials serve to be illustrative rather than exhaustive.

We count ourselves very lucky to have had so much time, energy, and patience from all of our contributors and have tried our best to use this wisely. We hope that - in addition to being a useful reference source - the book does something to document the passion and enthusiasm, the vision and energy of a nascent movement. The book attempts to give a sense of what happens behind the scenes, the stories behind the stories.

The Data Journalism Handbook is a work in progress. If you think there is anything which needs to be amended or is conspicuously absent, then please flag it for inclusion in the next version. It is also freely available under a Creative Commons Attribution-ShareAlike license, and we strongly encourage you to share it with anyone that you think might be interested in reading it.

Jonathan Gray (@jwyg)
Liliana Bounegru (@bb_liliana)
Lucy Chambers (@lucyfedia)
March 2012

Written by Khadija Sharife

Everything has a paper trail, a lead that exposes the systemic underwire of a network, company, or person’s illicit or illegal activities. The trick is to find it.

Recently, the African Network of Centers for Investigative Reporting (ANCIR) investigated a global Ponzi scheme controlled by a U.K.-based director, Renwick Haddow. He was the man at the top of an entity called Capital Organisation, which used a network of more than 30 shell companies to sell more than $180 million in fraudulent investments over five years.

It was a global network of interconnected entities, and our organization had a total budget of $500 to investigate and expose it. That budget was entirely invested in our Sierra Leone journalist who was needed to visit a farm related to the scam, meet the locals, and to extract documents from the relevant ministries. That left us with zero budget for other aspects of the story, including the financial trail.

How did we unravel the scam? By finding and following the paper trail, which in this case involved accessing a range of information from databases, corporate brochures, court records and other publicly available sources. All of the evidence we gathered is accessible here, and you can read our full investigation, “Catch and Release”, in the Spring 2015 issue of World Policy Journal.

Anatomy of a scam

The scam used the shell companies to peddle fabricated investments in far-off locations to investors, particularly U.K. pensioners. The purported investments ranged from agricultural (farms producing palm oil, rice, cocoa and wheat) to minerals (gold, platinum, diamonds) as well as properties, water bonds, Voice Over Internet Protocol, and more. High returns were promised, often with guaranteed exit strategies, which assured investors they could recoup their money with a profit.

Shell entities with names such as Agri Firma, Capital Carbon Credits and Voiptel International had no staff, bank accounts, offices or other components of real business. Instead, Haddow and his crew channeled money to financial receiving agents who then deposited it into tax havens such as Cyprus. Then final remittance was made to British Virgin Islands holding companies such as Rusalka and Glenburnie Investment.

The shell entities promoted investment schemes that were unregulated or lightly regulated by the U.K.’s Financial Conduct Authority (FCA). The investments were then promoted through fictitious brokers carrying names such as Capital Alternatives, Velvet Assets, Premier Alternatives, Able Alternatives and others. These entities were based in the U.K. and eventually spread around the world from Gibraltar to Dubai. They often consisted of nothing more than short-term or mailbox offices. Many even shared the same telephone number or address.

On the front line of the scam were often unscrupulous sales agents who were incentivized with commissions of between 25 and 40 percent of what they sold as new investments. The rest would be transferred as “investment arrangement fees” to the private offshore accounts of architects such as Renwick Haddow, Robert McKendrick and other key players.

Following the trail

The most important aspects of any investigation are to dig, listen and ask pertinent questions. But asking questions requires context, and listening to the right sources means finding the core of the story. Data, free or otherwise, can never replace good investigative research. In order to do good investigative research, these days, one must become familiar with how and where knowledge can be found, and how best to access and develop it.

Court documents showed us that this was not the first time that some of the people and entities in this scam had been investigated. Though the court document in question only looked at a seemingly minor question — whether it was a collective or individual scheme — the process often yields evidence and leads that may otherwise not be available.

We gathered corporate brochures that listed financial receiving agents, brokers, auditors, physical offices and other details that detailed connections between seemingly independent companies.

Our work made use of free public databases such as Duedil that allow for individual and corporate director searches. These enable users to identify the number of companies — current, dissolved, etc. — that a director is involved in. It can also provide other important information: Shareholders, registered offices and a timeline of retired and current individuals involved. We also used LinkedIn to probe prior personal and corporate connections.

Some free resources such as Duedil worked well for the U.K.-connected companies in this investigation. We followed up specific aspects with Companies House, Orbis and other corporate data sites, all of which are accessible for free to journalists via the Investigative Dashboard. The Dashboard “links to more than 400 online databases in 120 jurisdictions where you can search for information on persons and businesses of interest.”

The African Network of Centers for Investigative Reporting plays a role in coordinating the Dashboard’s Africa department. Unlike other jurisdictions, African countries often do not have digitized or electronically accessible data. To this end, we train and deploy in-country researchers to physically obtain not just the updated and accurate corporate, land, court and other data, but also to visit critical locations, conduct basic interviews and take relevant photos, among other things.

Along with databases, we used Whois Internet searches where possible to determine the date of creation and ownership information of websites that were connected to the network. We then cross- referenced the contact details of the websites with the information listed in corporate databases for the brokers and shell entities. Using specific search phrases, we were able to draw out mentions of certain names, companies, products etc. from various files on the Internet. We also searched for news articles about the people and companies identified in the network. We soon discovered that their ranks included murderers, money launderers and the like.

As part of the investigation, we also created dummy profiles on social media to enable us to connect with relevant companies and individuals, and to engage in email communication. We posed as potential investors to gain firsthand access to the push and pull of the scam.

A critical aspect of reporting was done in person. Once it was clear that Sierra Leone was a focal point of the story, we invested the $500 allocated from Open Society West Africa (OSIWA) to secure an in- country researcher, Silas Gbandia. He physically double-checked whether land leases were correctly entered, and if not, which sections or aspects were excluded.

Most investors in our story presumed the land leases were legitimate. Yet in all cases, the right to sublease by investors was not legal. Some land leases were not entered into the Sierra Leone official registry and therefore were not legitimate (such as those involving palm oil). At least one land lease was totally fraudulent; others were only partially legitimate. The use of in-country researchers to pull the registered land leases could not have more invaluable.

We used sourceAfrica, a free service by ANCIR, to annotate, redact and publish critical documents, including those sent to us by carefully cultivated and trusted sources.

Finally, with all of our information collected, we connected with Heinrich Böhmke, a South African prosecutor and an in-house expert at ANCIR, to “cross-examine” our evidence. This is a process Böhmke took from the legal world and adapted for investigative journalism. We looked for bias, contradictions, consistency and probability within evidence, resources, interviews and sources. A detailed guide to cross-examination for journalists is available here. (Along with Böhmke, we relied on Giovanni Pellerano, ANCIR’s in-house tech specialist, to help extract metadata from multiple electronic sources and documents.)

In the end, by identifying the broad relations within, and between, people, companies, jurisdictions, receiving agents and products, and by studying the corporate data from Duedil, Companies House and others, we were able to visualize the network’s structure. This told us how the scheme functioned and who was involved.

Much of this work was enabled by the analysis and investigation of publicly accessible information and documents. This data helped map the activity, people and entities in question and gave us the information we needed to further this investigation.

Key Questions

The bottom line is that it doesn’t take a genius to develop a good investigation or to lift the corporate veil — it simply takes curiosity, technique and a commitment to read as much and as far into the issue as possible. Scour as many data sources as possible: Corporate, media, NGO, shipping, sanctions, land... Look for what is not obvious, seems illogical, or that just plain sticks out to you. Follow your instinct. Ask as many questions as possible. For example, when investigating a corporate entity pursue questions such as:

• What does the company do?
  

• How many employees does it have? Who are they?
  

• In which countries does it operate?
  

• In which countries is it incorporated?
  

• What are the names of linked companies in each country of operation?

• Where does it pay taxes?
  

• Where does it report its profits?
  

• What is the extent of transfer pricing among its subsidiaries?
  

• Which companies use this practice and why? (And where?)

Remember, everything has a paper trail.

Written by Craig Silverman and Rina Tsubaki

“... There is a need on the part of all journalists to never assume anything and to always cross-check and verify in order to remain trusted sources of news and information.”

- Santiago Lyon, vice president and director of photography, The Associated Press

After an 8.1 magnitude earthquake struck northern India, it wasn’t long before word circulated that 4,000 buildings had collapsed in one city, causing “innumerable deaths.” Other reports said a college’s main building, and that of the region’s High Court, had also collapsed.

It was a similar situation when a 9.0 magnitude earthquake hit northeastern Japan. People heard that toxic rain would fall because of an explosion at an oil company’s facilities, and that it was not possible for aid agencies to air drop supplies within the country.

They were false rumors, every single one of them.

It’s a fundamental truth that rumors and misinformation accompany emergency situations. That earthquake in India? It occurred in 1934, long before the Internet and social media. The earthquake in Japan came in 2011.

Both quakes resulted in rumors because uncertainty and anxiety - two core elements of crises and emergency situations - cause people invent and repeat questionable information.

“In short, rumors arise and spread when people are uncertain and anxious about a topic of personal relevance and when the rumor seems credible given the sensibilities of the people involved in the spread,” write the authors of “Rumor Mills: The Social Impact of Rumor and Legend.”

An article in Psychology Today put it another way: “Fear breeds rumor. The more collective anxiety a group has, the more inclined it will be to start up the rumor mill.”

In today’s networked world, people also intentionally spread fake information and rumors as a joke, to drive “likes” and followers, or simply to cause panic.

As a result, the work of verification is perhaps most difficult in the very situations when providing accurate information is of utmost importance. In a disaster, whether its cause is natu- ral or human, the risks of inaccuracy are amplified. It can literally be a matter of life and death.

Yet amid the noise and hoaxes there is always a strong signal, bringing valuable, important information to light. When a US Airways flight was forced to land on the Hudson River, a man on a ferry was the source of an urgent, eye-opening image that only a bystander could have captured at that moment:

People on the ground are even more valuable in places where journalists have little or no access, and aid agencies have not been able to operate. Today, these witnesses and participants often reach for a phone to document and share what they see. It could be a bystander on a boat in a river - or a man who just walked away from a plane crash, as with this example from 2013:

The public relies on official sources such as news organizations, emergency services and government agencies to provide cred- ible, timely information.

But, at the same time, these organizations and institutions increasingly look to the public, the crowd, to help source new information and bring important perspective and context. When it works, this creates a virtuous cycle: Official and established sources of information - government agencies, NGOs, news organizations - provide critical information in times of need, and work closely with the people on the ground who are first to see and document an emergency.

To achieve this, journalists and humanitarian and emergency workers must become adept at using social media and other sources to gather, triangulate and verify the often conflicting information emerging during a disaster. They require proven processes, trustworthy tools, and tried and true techniques. Most of all, they need to gain all of the aforementioned before a disaster occurs.

A disaster is no time to try to verify on the fly. It’s not the moment to figure out what your standards and practices are for handling crowdsourced information. Yet it’s what many - too many - newsrooms and other organizations do.

Fortunately, an abundance of tools, technologies and best practices have emerged in recent years that enable anyone to master the new art of verification, and more are being developed all the time.

It is, in the end, about achieving a harmony of two core elements: Preparing, training and coordinating people in advance and during an emergency; and providing them with access and resources to enable them to take full advantage of the ever-evolving tools that can help with verification.

The combination of the human and the technological with a sense of direction and diligence is ultimately what helps speed and perfect verification. Admittedly, however, this is a new combination, and the landscape of tools and technologies can change quickly.

This book synthesizes the best advice and experience by drawing upon the expertise of leading practitioners from some of the world’s top news organizations, NGOs, volunteer and technical communities, and even the United Nations. It offers essential guidance, tools and processes to help organizations and professionals serve the public with reliable, timely information when it matters most.

The truth is that good professionals often fall for bad information, and that technology can lead us astray just as much as it can help. This can be even more true when so much information is moving at such a fast pace, and when so many newsrooms and organizations lack formal verification training programs and processes.

“The business of verifying and debunking content from the public relies far more on journalistic hunches than snazzy technology,” wrote David Turner in a Nieman Reports article about the BBC’s User Generated Content Hub. “While some call this new specialization in journalism ‘information forensics,’ one does not need to be an IT expert or have special equipment to ask and answer the fundamental questions used to judge whether a scene is staged or not.”

This realization that there is no silver bullet, no perfect test, is the starting point for any examination of verification, and for the work of providing reliable information in a disaster. This requires journalists and others to first look to the fundamentals of verification that have existed for decades and that won’t become obsolete.

Steve Buttry focuses on a core question at the heart of verification in his chapter. Joining that is this list of fundamentals:

• Put a plan and procedures in place before disasters and breaking news occurs.
• Develop human sources.
• Contact people, talk to them.
• Be skeptical when something looks, sounds or seems too good to be true.
• Consult credible sources.
• Familiarize yourself with search and research methods, and new tools.
• Communicate and work together with other professionals - verification is a team sport.

One other maxim that has been added to the above list in recent years is that when trying to evaluate information - be it an image, tweet, video or other type of content - you must verify the source and the content.

When The Associated Press promoted Fergus Bell to take the lead in creating and practicing its process for confirming user-generated video, he first looked to the organization’s long- standing guidance on verification, rather than to new tools and technology.

“AP has always had its standards and those really haven’t changed, and it was working with those standards that we were able to specifically set up workflows and best practices for dealing with social media,” Bell said. “So AP has always strived to find the original source so that we can do the reporting around it. And that’s always the way that we go about verifying UGC. We can’t verify something unless we speak to the person that created it, in most cases.”

By starting with these fundamentals, organizations can begin to build a reliable, repeatable process for verifying information during emergency situations. Verifying information on social networks, be it claims of fact, photos or video, becomes easier once you know your standards, and know how to apply them.

That’s when it’s possible to make the best use of tools such as EXIF readers, photo analysis plug-ins, advanced Twitter search, whois domain lookups and the other tools outlined in this book.

Along with that toolkit, and the standards and processes that inform how we use the tools, there is also the critical element of crowdsourcing: bringing the public into the process and working with them to ensure we all have better information when it matters most.

Andy Carvin, who recently left the job of senior social strategist at NPR, is perhaps the most celebrated and experienced practitioner of crowdsourced verification. He said the key is to work with the crowd to, as the NPR motto goes, “create a more informed public.”

“When a big story breaks, we shouldn’t just be using social media to send out the latest headlines or ask people for their feedback after the fact,” he said in a keynote address at the International Journalism Festival.

He continued:

We shouldn’t even stop at asking for their help when trying to cover a big story. We should be more transparent about what we know and don’t know. We should actively address rumors being circulated online. Rather than pretending they’re not circulating, or that they’re not our concern, we should tackle them head-on, challenging the public to question them, scrutinize them, understand where they might have come from, and why.

This book is a guide to help all of us - journalists, emergency responders, citizen reporters and everyone else - gain the skills and knowledge necessary to work together during critical events to separate news from noise, and ultimately to improve the quality of information available in our society, when it matters most.

Written by Giannina Segnini

Never before have journalists had so much access to information. More than three exabytes of data — equivalent to 750 million DVDs — are created every day, and that number duplicates every 40 months. Global data production is today being measured in yottabytes. (One yottabye is equivalent to 250 trillion DVDs of data.) There are already discussions underway about the new measurement needed once we surpass the yottabyte.

The rise in the volume and speed of data production might be overwhelming for many journalists, many of whom are not used to using large amounts of data for research and storytelling. But the urgency and eagerness to make use of data, and the technology available to process it, should not distract us from our underlying quest for accuracy. To fully capture the value of data, we must be able to distinguish between questionable and quality information, and be able to find real stories amid all of the noise.

One important lesson I’ve learned from two decades of using data for investigations is that data lies — just as much as people, or even more so. Data, after all, is often created and maintained by people.

Data is meant to be a representation of the reality of a particular moment of time. So, how do we verify if a data set corresponds to reality?

Two key verification tasks need to be performed during a data-driven investigation: An initial evaluation must occur immediately after getting the data; and findings must be verified at the end of the investigation or analysis phase.

A. Initial verification

The first rule is to question everything and everyone. There is no such thing as a completely reliable source when it comes to using data to make meticulous journalism.

For example, would you completely trust a database published by the World Bank? Most of the journalists I’ve asked this question say they would; they consider the World Bank a reliable source. Let’s test that assumption with two World Bank datasets to demonstrate how to verify data, and to reinforce that even so-called trustworthy sources can provide mistaken data. I’ll follow the process outlined in the below graphic.

1. Is the data complete?

One first practice I recommend is to explore the extreme values (highest or lowest) for each variable in a dataset, and to then count how many records (rows) are listed within each of the possible values.

For example, the World Bank publishes a database with more than 10,000 independent evaluations performed on more than 8,600 projects developed worldwide by the organization since 1964.

Just by sorting the Lending Cost column in ascending order in a spreadsheet, we can quickly see how multiple records have a zero in the cost column.

If we create a pivot table to count how many projects have a zero cost, in relation to the total records, we can see how more than half of those (53 percent) cost zero.

This means that anyone who performs a calculation or analysis per country, region or year involving the cost of the projects would be wrong if they failed to account for all of the entries with no stated cost. The dataset as it’s provided will lead to an inaccurate conclusion.

The Bank publishes another database that supposedly contains the individual data for each project funded (not only evaluated) by the organization since 1947.

Just by opening the api.csv file in Excel (version as of Dec. 7, 2014), it’s clear that the data is dirty and contains many variables combined into one cell (such as sector names or country names). But even more notable is the fact that this file does not contain all of the funded projects since 1947.

The database in fact only includes 6,352 out of the more than 15,000 projects funded by the World Bank since 1947. (Note: The Bank eventually corrected this error. By Feb. 12, 2015, the same file included 16,215 records.)

After just a little bit of time spent examining the data, we see that the World Bank does not include the cost of all projects in its databases, it publishes dirty data, and it failed to include all of its projects in at least one version of the data. Given all of that, what would you now expect about the quality of data published by seemingly less reliable institutions?

Another recent example of database inconsistency I found came during a workshop I was giving in Puerto Rico for which we used the public contracts database from the Comptroller’s Office. Some 72 public contracts, out of all last year’s contracts, had negative values ($–10,000,000) in their cost fields.

Open Refine is an excellent tool to quickly explore and evaluate the quality of databases.
In the first image below, you can see how Open Refine can be used to run a numeric “facet” in the Cuantía (Amount) field. A numeric facet groups numbers into numeric range bins. This enables you to select any range that spans a consecutive number of bins.

The second image below shows that you can generate a histogram with the values range included in the database. Records can then be filtered by values by moving the arrows inside the graph. The same can be done for dates and text values.

2. Are there duplicate records?

One common mistake made when working with data is to fail to identify the existence of duplicate records.

Whenever processing disaggregated data or information about people, companies, events or transactions, the first step is to search for a unique identification variable for each item. In the case of the World Bank’s projects evaluation database, each project is identified through a unique code or “Project ID.” Other entities’ databases might include a unique identification number or, in the case of public contracts, a contract number.

If we count how many records there are in the database for each project, we see that some of them are duplicated up to three times. Therefore, any calculation on a per country, region or date basis using the data, without eliminating duplicates, would be wrong.

In this case, records are duplicated because multiple evaluation types were performed for each one. To eliminate duplicates, we have to choose which of all the evaluations made is the most reliable. (In this case, the records known as Performance Assessment Reports [PARs] seem to be the most reliable because they offer a much stronger picture of the evaluation. These are developed by the Independent Evaluation Group, which independently and randomly samples 25 percent of World Bank projects per year. IEG sends its experts to the field to evaluate the results of these projects and create independent evaluations.)

3. Are the data accurate?

One of the best ways to assess a dataset’s credibility is to choose a sample record and compare it against reality.

If we sort the World Bank’s database — which supposedly contained all the projects developed by the institution — in descending order per cost, we find a project in India was the most costly. It is listed with a total amount of US$29,833,300,000.

If we search the project’s number on Google (P144447), we can access the original approval documentation for both the project and its credit, which effectively features a cost of US$29,833 million. This means the figure is accurate.

It’s always recommended to repeat this validation exercise on a significant sample of the records.

4. Assessing data integrity

From the moment it’s first entered in a computer to the time when we access it, data goes through several input, storage, transmission and registry processes. At any stage, it may be manipulated by people and information systems.

It’s therefore very common that relations between tables or fields get lost or mixed up, or that some variables fail to get updated. This is why it’s essential to perform integrity tests.

For example, it would not be unusual to find projects listed as “active” in the World Bank’s database many years after the date of approval, even if it’s likely that many of these are no longer active.

To check, I created a pivot table and grouped projects per year of approval. Then I filtered the data to show only those marked as “active” in the “status” column. We now see that 17 projects approved in 1986, 1987 and 1989 are still listed as active in the database. Almost all of them are in Africa.

In this case, it’s necessary to clarify directly with the World Bank if these projects are still active after almost 30 years.

We could, of course, perform other tests to evaluate the World Bank’s data consistency. For example, it would be a good idea to examine whether all loan recipients (identified as “borrowers” in the database) correspond to organizations and/or to the actual governments from the countries listed in the “Countryname” field, or whether the countries are classified within the correct regions (“regionname”).

5. Deciphering codes and acronyms

One of the best ways to scare a journalist away is to show him or her complex information that’s riddled with special codes and terminology. This is a preferred trick by bureaucrats and organizations who offer little transparency. They expect that we won’t know how to make sense of what they give us. But codes and acronyms can also be used to reduce characters and leverage storage capacities. Almost every database system, either public or private, uses codes or acronyms to classify information.

In fact, many of the people, entities and things in this world have one or several codes assigned. People have identification numbers, Social Security numbers, bank client numbers, taxpayer numbers, frequent flyer numbers, student numbers, employee numbers, etc.

A metal chair, for example, is classified under the code 940179 in the world of international commerce. Every ship in the world has a unique IMO number. Many things have a single, unique number: Properties, vehicles, airplanes, companies, computers, smartphone, guns, tanks, pill, divorces, marriages...

It is therefore mandatory to learn how to decrypt codes and to understand how they are used to be able to understand the logic behind databases and, more importantly, their relations.

Each one of the 17 million cargo containers in the world has a unique identifier, and we can track them if we understand that the first four letters of the identifier are related to the identity of its owner. You can query the owner in this database. Now those four letters of a mysterious code become a means to gain more information.

The World Bank database of evaluated projects is loaded with codes and acronyms and, surprisingly, the institution does not publish a unified glossary describing the meaning of all these codes. Some of the acronyms are even obsolete and cited only in old documents.

The “Lending Instrument” column, for example, classifies all projects depending on 16 types of credit instruments used by the World Bank to fund projects: APL, DPL, DRL, ERL, FIL, LIL, NA, PRC, PSL, RIL, SAD, SAL, SIL, SIM, SSL and TAL. To make sense of the data, it’s essential to research the meaning of these acronyms. Otherwise you won’t know that ERL corresponds to emergency loans given to countries that have just undergone an armed conflict or natural disaster.

The codes SAD, SAL, SSL and PSL refer to the disputed Structural Adjustment Program the World Bank applied during the ’80s and ’90s. It provided loans to countries in economic crises in exchange for those countries’ implementation of changes in their economic policies to reduce their fiscal deficits. (The program was questioned because of the social impact it had in several countries.)

According to the Bank, since the late ’90s it has been more focused on loans for “development,” rather than on loans for adjustments. But, according to the database, between the years 2001 and 2006, more than 150 credits were approved under the Structural Adjustment code regime.

Are those database errors, or has the Structural Adjustment Program been extended into this century?

This example shows how decoding acronyms is not only a best practice for evaluating the quality of the data, but, more important, to finding stories of public interest.

B. Verifying data after the analysis

The final verification step is focused on your findings and analysis. It is perhaps the most important verification piece, and the acid test to know if your story or initial hypothesis is sound.

In 2012, I was working as an editor for a multidisciplinary team at La Nación in Costa Rica. We decided to investigate one of the most important public subsidies from the government, known as “Avancemos.” The subsidy paid a monthly stipend to poor students in public schools to keep them from leaving school.

After obtaining the database of all beneficiary students, we added the names of their parents. Then we queried other databases related to properties, vehicles, salaries and companies in the country. This enabled us to create an exhaustive inventory of the families’ assets. (This is public data in Costa Rica, and is made available by the Supreme Electoral Court.)

Our hypothesis was that some of the 167,000 beneficiary students did not live in poverty conditions, and so should not have been receiving the monthly payment.

Before the analysis, we made sure to evaluate and clean all of the records, and to verify the relationships between each person and their assets.

The analysis revealed, among other findings, that the fathers of roughly 75 students had monthly wages of more than US$2,000 (the minimum wage for a nonskilled worker in Costa Rica is $500), and that over 10,000 of them owned expensive properties or vehicles.

But it was not until we went to visit their homes that we could prove what the data alone could have never told us: These kids lived in real poverty with their mothers because they had been abandoned by their fathers.

No one ever asked about their fathers before granting the benefit. As a result, the state financed, over many years and with public funds, the education of many children who had been abandoned by an army of irresponsible fathers .

This story summarizes the best lesson I have learned in my years of data investigations: Not even the best data analysis can replace on-the-ground journalism and field verification.

Written by Stéphanie Durand

The region of Jos in Central Nigeria is traditionally known as the “Home of Peace and Tourism.” Today, and for some time now, it is home to an ongoing war along religious and sectarian lines.

Jos straddles the north-south border of Nigeria. The northern part of the country is predominantly Muslim; the south is predominantly Christian.

The crisis in Jos has led to alarming headlines such as “Islamic Assailants Kill Hundreds of Christians near Jos” and “Muslims Slaughter Christians in Central Nigeria.” Those headlines and others like it prompted some religious leaders to blame the media for inciting religious violence because of the provocative nature of the reports.

But there is deadly violence in Jos and the press must accurately tell that story. To do so, they must sift through an increasing number of rumors that spread via text messages, social media and blogs - and be careful to avoid publishing false information that further enflames the situation.

Local journalists are also exposed to intimidation, self-censorship and fear of retribution from state authorities or militants. International media face challenges from decreasing resources that result in foreign reporters’ working alone to cover an entire region

This can affect their knowledge of the local context and sensitivity to it. It also increases their reliance on content gathered and distributed by (often unknown) witnesses on the ground.

Journalists must be careful to verify what they discover, or risk increasing tensions and generating reprisal attacks based on nothing more than rumors.

In January 2010, when news outlets started reporting another major conflict in Jos, rumors began to spread about mobs armed with knives and machetes around houses, mosques and churches. Witnesses reported different causes of the conflict: Some said it was because of the rebuilding of houses destroyed by riots in 2008, others a fight during a football match, or the burning of a church.

Text messages also played a significant role in directly inciting violence with messages such as “slaughter them before they slaughter you. Kill them before they kill you.”

At the same time, blogs regularly displayed photos of the victims of violence.

The verification process is more crucial than ever in a situation where misperception and fear pervade all sides. It is essential for journalists to remove themselves from the passions of those involved, and verify the accuracy of accounts that narrate or visually feature ethnic or religious violence. Debunking a false rumor about a murderous rampage, or impending crisis, can literally save lives.

As is the case elsewhere, in Jos social media perpetuate misinformation, while at the same time enabling journalists to connect and interact with members of the public as part of their work. Social media also provide a platform to respond to rumors, and verify information that ultimately creates the type of trust and transparency necessary to avoid an escalation of conflict.

In Jos, the application of verification, in collaboration with the public, helps the media play a role in diffusing tension and containing conflict. It results in, and encourages, fair and accurate reporting that is sorely needed.

While this is certainly not the only response needed to alleviate tensions, such reporting goes a long way towards dissipating the fear, suspicion and anger that is at the heart of ethnic and religious conflicts.

Abstract

How we used data to reveal illegal business practices, sustained environmental damage and slave-like conditions for workers in developing countries’ agroindustries.

Keywords: cross-border investigations, agriculture, colonialism, data journalism, environmental damage

At the beginning of 2016, a small group of journalists decided to investigate the journey of a chocolate bar, banana and cup of coffee from the original plantations to their desks.

Our investigation was prompted by reports that all of these products were produced in poor countries and mostly consumed in rich countries.

Starting from that data we decided to ask some questions:

-What are the labour conditions on these plantations like?

-Is there a concentration of land ownership by a small group?

-What kinds of environmental damage do these products cause in these countries?

So El Diario and El Faro (two digital and independent media outlets in Spain and El Salvador) joined forces to investigate the dark side of the agroindustry business model in developing countries.1

The resulting “Enslaved Land” project is a one-year cross-border and data-driven investigation that comes with a subheading that gets straight to the point: “This is how poor countries are used to feed rich countries”.²

In fact, colonialism is the main issue of this project. As journalists, we didn’t want to tell the story of the poor indigenous people without examining a more systemic picture.

We wanted to explain how land property, corruption, organized crime, local conflicts and supply chains of certain products are still part of a system of colonialism.

In this project, we investigated five crops consumed widely in Europe and the US: sugar, coffee, cocoa, banana and palm oil in Guatemala, Colombia, Ivory Coast and Honduras. As a data driven investigation, we used the data to get from pattern to story.

The choice of crops and countries was made based on a previous data analysis of 68 million records of United Nations World Trade Database (Fig. 1.1).

This investigation shows how balance of power between rich and poor countries has changed from the 15th century to present and prove that these crops are produced thanks to exploitative, slave-like conditions for workers, illegal business practices and sustained environmental damage.

The focus of our stories was shaped by the data we used. In Honduras, the key was to use geographic information to tell the story. We compiled the land use map of the country and overlaid the surface of palm plantations with protected areas. We found that 7,000 palm oil hectares were illegally planted in protected areas of the country.

As a result, our reporter could investigate the specific zones with palm plantations in protected areas. The story uses individual cases to highlight and narrate systemic abuse, such as the case of Monchito, a Honduran peasant who grows African palm in the Jeannette Kawas National Park.

This project is not only about land use. In Guatemala, we created a database of all the sugar mills in the country. We dived into the local company registry to find out the owners and directors of the mills. Next we used public business records to link these individuals and entities with offshore companies in Panama, Virgin Islands and the Bahamas.

To find out how they create and manage the offshore structure, El Faro had access to the Panama Papers database, so we used that information to reconstruct how one of the biggest mills of the country worked with the Mossack Fonseca law firm to avoid taxes.

A transnational investigation aiming to uncover corruption and business malpractice in poor countries is challenging in many ways. We had to work in rural areas where there is no governmental presence, and in most cases the reporting posed some risk. We dealt with countries where there is a considerable lack of transparency, where open data is absent, and, in some cases, where public administrations do not know what information they hold.

Honduras and Guatemala were only one aspect of our investigation. More than 10 people worked together to produce this material. All this work was coordinated from the offices of El Diario in Spain and El Faro in El Salvador, working alongside journalists in Colombia, Guatemala, Honduras and Ivory Coast.

This work was undertaken not only by journalists, but also by editors, photographers, designers and developers who participated in the development and production process to develop an integrated web product. This project would not have been possible without them.

We used an integrated scrollytelling narrative for each of the investigations. For us, the way that users read and interact with the stories is as important as the investigation itself. We chose to combine satellite images, photos, data visualizations and narrative because we wanted the reader to understand the link between the products they consumed and the farmers, companies, and other actors involved in their production.

This structure allowed us to combine personal stories with data analysis in a compelling narrative. One example is the story of John Pérez, a Colombian peasant whose land was stolen by paramilitary groups and banana corporations during the armed conflict. To tell this story we used a zoomable map that takes you from his plantation to the final destination of Colombian banana production.

This project showed that data journalism can enrich traditional reporting techniques to connect stories about individuals to broader social, economic and political contexts.

Our investigation was also published by Plaza Pública in Guatemala and Ciper in Chile, and was included in the Guatemalan radio show “ConCriterio.” The latter led to a public statement from the Guatemalan Tax Agency asking for resources to fight against tax fraud in the sugar mill business.

Footnotes

1. www.eldiario.es (Spanish language), elfaro.net (Spanish language)

2. latierraesclava.eldiario.es (Spanish language)

Infographic impresario Lulu Pinney created this superb poster, which gives an overview of the contents of the Data Journalism Handbook.

Written by Eliot Higgins

During the later stages of the Libyan civil war in 2011, rebel groups pushed out from the Nafusa Mountain region and began to capture towns. There were many contradictory reports of the capture of towns along the base of the mountain range. One such claim was made about the small town of Tiji, just north of the mountains. A video was posted online that showed a tank driving through what was claimed to be the center of the town.

At the time, I was examining user-generated content coming from the Libyan conflict zone. My interest was in understanding the situation on the ground, beyond what was being reported in the press. There were constant claims and counterclaims about what was happening on the ground. There was really only one question I was interested in answering: How do we know if a report is accurate?

This is why and how I first learned to use geolocation to verify the location where videos were filmed. This work helped me sharpen the open source investigation techniques that are now used by myself and others to investigate everything from international corruption to war zones and plane crashes.

The video in Tiji showed a tank driving down a wide road, right next to a mosque. Tiji was a small town; I thought it might be easy to find that road and the mosque.

Until that point, I hadn’t even considered that you could use satellite maps to look for landmarks visible in videos to confirm where they had been filmed. The satellite map imagery below clearly showed only one major road running through the town, and on that road there was one mosque. I compared the position of the minaret, the dome and a nearby wall on the satellite map imagery to that in the video, and it was clear it was a perfect match.

Now that the likely position of the camera in the town was established, I could watch the whole video, comparing other details to what was visible on satellite map imagery. This further confirmed the positions matched.

Building expertise in satellite map based geolocation was something I did over time, using new tricks and techniques as I moved onto new videos.

Matching roads

After the Tiji video, I examined a video purportedly filmed in another Libyan town, Brega, which featured rebel fighters taking a tour of the streets. At first, it appeared there were no large features, such as mosques, on a satellite map imagery. But I realized there was one very large feature visible in the video. As they walked through the streets, it was possible to map out the roads along the route they took, and then match that pattern to what was visible in satellite map imagery. Below is a hand-drawn map of the roads, as I saw them represented on the video.

I scanned the satellite imagery of the town, looking for a similar road pattern. I soon found a match:

Hunting shadows

As you become more familiar with geolocating based on satellite map imagery, you’ll learn how to spot smaller objects as well. For example, while things like billboards and streetlights are small objects, the shadows they cast can actually indicate their presence. Shadows can also be used to reveal information about the comparative height of buildings, and the shape of those buildings:

Shadows can also be used to tell the time of day an image was recorded. After the downing of Flight MH17 in Ukraine, the following image was shared showing a Buk missile launcher in the town of Torez:

It was possible to establish the exact position of the camera, and from that, it was possible to establish the direction of the shadows. I used the website Sun Calc, which allows users to calculate the position of the sun throughout the day using a Google Maps based interface. It was then possible to establish the time of day as approximately 12:30 p.m. local time, which was later supported by interviews with civilians on the ground, and with social media sightings of the missile launcher traveling through the town.

In the case of July 17, 2014, and the downing of MH17, it was possible to do this by analyzing several videos and photographs of the Buk missile launcher. I and others were able to create a map of the missile launcher’s movements on the day, as well as a timeline of sightings.

By bringing together different sources, tools and techniques, it was possible to connect these individual pieces of information and establish critical facts about this incident.

A key element of working with user-generated content in investigations is understanding how that content is shared. With Syria, a handful of opposition social media pages are the main sources of information from certain areas. This obviously limits the perspective on the conflict from different regions, but also means it’s possible to collect, organize and systematically review those accounts for the latest information.

In the case of Ukraine, there’s few limits on Internet access, so information is shared everywhere. This creates new challenges for collecting information, but it also means there’s more unfiltered content that may contain hidden gems.

During Bellingcat’s research on the Buk missile launcher linked to the downing of MH17, it was possible to find multiple videos of a convoy traveling through Russia to the Ukrainian border that had the same missile launcher filmed and photographed on July 17 inside Ukraine.

These videos were on social media accounts and several different websites, all of which belonged to different individuals. They were uncovered by first geolocating the initial videos we found, then using that to predict the likely route those vehicles would have taken to get from each geolocated site. Then we could keyword search on various social media sites for the names of locations that were along the route the vehicle would have to had to travel. We also searched for keywords such as “convoy,” “missile,” etc. that could be associated with sightings.

Although this was very time consuming, it allowed us to build a collection of sightings from multiple sources that would have otherwise been overlooked, and certainly not pieced together.

If there’s one final piece of advice, it would be to give this work and approach a try in any investigation. It’s remarkable what you can turn up when you approach UGC and open source information in a systematic way. You tend to learn quickly by just doing it. Even something as simple as double- checking the geolocation someone else has done can teach you a lot about comparing videos and photographs to satellite map imagery.

Claire Wardle leads the strategic direction and research for First Draft, a global nonprofit that supports journalists, academics and technologists working to address challenges relating to trust and truth in the digital age. She has been a Fellow at the Shorenstein Center for Media, Politics and Public Policy at Harvard's Kennedy School, the Research Director at the Tow Center for Digital Journalism at Columbia University's Graduate School of Journalism and head of social media for UNHR, the United Nations Refugee Agency.

As we all know, lies, rumors and propaganda are not new concepts. Humans have always had the ability to be deceptive, and there are some glorious historical examples of times when fabricated content was used to mislead the public, destabilize governments or send stock markets soaring. What’s new now is the ease with which anyone can create compelling false and misleading content, and the speed with which that content can ricochet around the world.

We’ve always understood that there was complexity in deception. One size does not fit all. For example, a white lie told to keep the peace during a family argument is not the same as a misleading statement by a politician trying to win over more voters. A state-sponsored propaganda campaign is not the same as a conspiracy about the moon landing.

Unfortunately, over the past few years, anything that might fall into the categories described here has been labeled “fake news,” a simple term that has taken off globally, often with no need for translation.

I say unfortunate, because it is woefully inadequate to describe the complexity we’re seeing. Most content that is deceptive in some way does not even masquerade as news. It is memes, videos, images or coordinated activity on Twitter, YouTube, Facebook or Instagram. And most of it isn’t fake; it’s misleading or, more frequently, genuine, but used out of context.

The most impactful disinformation is that which has a kernel of truth to it: taking something that is true and mislabeling it, or sharing something as new when actually it’s three years old.

Perhaps most problematic is that the term fake news has been weaponized, mostly by politicians and their supporters to attack the professional news media around the world.

My frustration at the phrase led me to coin the term “information disorder” with my co-author Hossein Derakhshan. We wrote a report in 2017 entitled “Information Disorder,” and explored the challenges of the terminology that exists on this topic. In this chapter, I will explain some of the key definitional aspects to understanding this subject, and critically talking about it.

7 Types of Information Disorder

Back in 2017, I created the following typology to underscore the different types of information disorder that exist.

Satire/Parody

Understandably, many people have pushed back against my including satire in this typology, and I certainly struggled with including this category. But unfortunately, agents of disinformation deliberately label content as satire to ensure that it will not be “fact-checked,” and as a way of excusing any harm that comes from the content. In an informational ecosystem, where context and cues, or mental shortcuts (heuristics) have been stripped away, satirical content is more likely to confuse the reader. An American might know that The Onion is a satirical site, but did you know that, according to Wikipedia, there are 57 satirical news websites globally? If you don’t know the website is satirical, and it’s speeding past you on a Facebook feed, it’s easy to be fooled.

Recently, Facebook took the decision not to fact-check satire, but those who work in this space know how the satire label is used as a deliberate ploy. In fact, in August 2019, the U.S. debunking organization Snopes wrote a piece about why they fact-check satire. Content purporting to be satire will evade the fact-checkers, and frequently over time, the original context gets lost: people share and re-share not realizing the content is satire and believing that it is true.

False Connection

This is old-fashioned clickbait: the technique of making claims about content via a sensational headline, only to find the headline is horribly disconnected from the actual article or piece of content. While it’s easy for the news media to think about the problem of disinformation as being caused by bad actors, I argue that it’s important to recognize that poor practices within journalism add to the challenges of information disorder.

Misleading Content

This is something that has always been a problem in journalism and politics. Whether it’s the selection of a partial segment from a quote, creating statistics that support a particular claim but don’t take into account how the data set was created, or cropping a photo to frame an event in a particular way, these types of misleading practices are certainly not new.

False Context

This is the category where we see the most content: It almost always occurs when genuine imagery is re-shared as new. It often happens during a breaking news event when old imagery is re-shared, but it also happens when old news articles are re-shared as new, when the headline still potentially fits with contemporary events.

Imposter Content

This is when the logo of a well-known brand or name is used alongside false content. This tactic is strategic because it plays on the importance of heuristics. One of the most powerful ways we judge content is if it has been created by an organization or person that we already trust. So by taking a trusted news organization’s logo and adding it to a photo or a video, you’re automatically increasing the chance that people will trust the content without checking.

Manipulated Content

This is when genuine content is tampered with or doctored in some way. The video of Nancy Pelosi from May 2019 is an example of this. The Speaker of the U.S. House of Representatives was filmed giving a speech. Just a few hours later, a video emerged of her speaking that made her sound drunk. The video had been slowed down, and by doing so, it made it appear like she was slurring her words. This is a powerful tactic, because it’s based on genuine footage. If people know she gave that speech with that backdrop, it makes them more trusting of the output.

Fabricated Content

This category is for when content is 100% fabricated. This might be making a completely new fake social media account and spreading new content from it. This category includes deepfakes, where artificial intelligence is used to manufacture a video or audio file in which someone is made to say or do something that they never did.

Understanding Intent and Motivation

These types are useful for explaining the complexity of the polluted information environment, but it doesn’t tackle the question of intent. This is a crucial part of understanding this phenomenon.

To do that, Derakhshan and I created this Venn diagram as a way of explaining the difference between misinformation, disinformation and a third term we created, malinformation. Misinformation and disinformation are both examples of false content. But disinformation is created and shared by people who hope to do harm, whether that’s financial, reputational, political or physical harm. Misinformation is also false, but people who share the content don’t realize it’s false. This is often the case during breaking news events when people share rumors or old photos not realizing that they’re not connected to the events.

Malinformation is genuine information, but the people who share it are trying to cause harm. The leaking of Hillary Clinton’s emails during the 2016 U.S. presidential election is an example of that. So is sharing revenge porn.

These terms matter, as intent is part of how we should understand a particular piece of information. There are three main motivations for creating false and misleading content. The first is political, whether foreign or domestic politics. It might be a case of a foreign government’s attempting to interfere with the election of another country. It might be domestic, where one campaign engages in “dirty” tactics to smear their opponent. The second is financial. It is possible to make money from advertising on your site. If you have a sensational, false article with a hyperbolic headline, as long as you can get people to click on your URL, you can make money. People on both sides of the political spectrum have talked about how they created fabricated “news” sites to drive clicks and therefore revenue. Finally, there are social and psychological factors. Some people are motivated simply by the desire to cause trouble and to see what they can get away with; to see if they can fool journalists, to create an event on Facebook that drives people out on the streets to protest, to bully and harass women. Others end up sharing misinformation, for no other reason than their desire to present a particular identity. For example, someone who says, “I don’t care if this isn’t true, I just want to underline to my friends on Facebook, how much I hate [insert candidate name].”

The Trumpet of Amplification

To truly understand this wider ecosystem, we need to see how intertwined it all is. Too often, someone sees a piece of misleading or false content somewhere, and believes it was created there. Unfortunately, those who are most effective when it comes to disinformation understand how to take advantage of its fragmented nature.

Remember also, that if rumors, conspiracies or false content weren’t shared, they would do no harm. It’s the sharing that is so damaging. I therefore created this image, which I call the Trumpet of Amplification, as a way of describing how agents of disinformation use coordination to move information through the ecosystem.

Too often, content is posted in spaces like 4Chan or Discord (an app used by gamers to communicate). These spaces are anonymous and allow people to post without recourse. Often these spaces are used to share specific details about coordination, such as "we’re going to try to get this particular hashtag to trend,” or “use this meme to respond to today’s events on Facebook.”

The coordination often then moves into large Twitter DM groups or WhatsApp groups, where nodes within a network spread content to a wider group of people. It might then move into communities on sites like Gab, Reddit or YouTube. From there, the content will often be shared into more mainstream sites like Facebook, Instagram or Twitter.

From there, it will often get picked up by the professional media, either because they don’t realize the provenance of the content and decide to use it in their reporting, without sufficient checks, or they decide to debunk the content. Either way, the agents of disinformation see it as a success. Poor headlines where the rumor or misleading claim is reported, or debunks where the false content is embedded in the story, play into the original plan: to drive amplification, to fan the rumor with oxygen.

At First Draft, we talk about the concept of the tipping point. For journalists, reporting on falsehoods too early provides additional and potentially damaging oxygen to a rumor. Reporting too late means it has taken hold and there is little that can be done. Working out that tipping point is challenging. It differs by location, topic and platform.

Conclusion

Language matters. This phenomenon is complex and the words we use makes a difference. We already have academic research that shows that increasingly audiences equate the description “fake news” with poor reporting practices from the professional media.

Describing everything as disinformation, when it might not actually be false content, or is being shared unknowingly by people who don’t think is false, are other crucial elements of understanding what is happening.

We live in an age of information disorder. It is creating new challenges for journalists, researchers and information professionals. To report or not to report? How to word headlines? How to debunk videos and images effectively? How to know when to debunk? How does one measure the tipping point? They are all new challenges that exist today for those working in the information environment. It’s complicated.

Written by Aaron Williams

Abstract

Visualizing racial segregation in the US with census data.

Keywords: programming, mapping, racial segregation, census, data visualization, data journalism

How do you measure segregation by race?

In the United States in particular, there has been a historical effort to separate people since its founding. As the country changed, and racist policies like segregation were outlawed, new laws emerged that aimed to keep African Americans as well as other groups separate from White Americans. Many Americans have experienced the lingering effects of these laws, but I wanted to know if there was a way to measure the impact based on where people live.

I was inspired after reading We Gon’ Be Alright: Notes on Race and Resegregation by Jeff Chang, a book of essays where the author explores the connecting themes of race and place. I was struck by chapters that talked about the demographic changes of places like San Francisco, Los Angeles and New York City and wanted to work on a project that quantified the ideas Chang wrote about.

Many maps that show segregation actually don’t. These maps often show a dot for each member of a specific race or ethnicity within a geography and colour that dot by the person’s race. They end up showing fascinating population maps about where people live but do not measure how diverse or segregated these areas are.

How do we know this? Well, segregation and diversity are two terms that have wildly different definitions depending on who you talk to. And while many people may perceive where they live as segregated, that answer can change depending on how one measures segregation. I didn’t want to act on anecdote alone.

Thus, I looked for ways to measure segregation in an academic sense and base my reporting from there.

I interviewed Michael Bader, an associate professor of sociology at American University in Washington, DC, who showed me the Multigroup Entropy Index (or Theil Index), a statistical measure that determines the spatial distribution of multiple racial groups simultaneously. We used this to score every single census block group in the United States compared to the racial population of the county it inhabited.

This project took roughly a year to complete. Most of the time before then was spent exploring the data and various measures of segregation.

During my research, I learned that there are several ways to measure segregation. For example, the Multigroup Entropy Index is a measure of evenness, which compares the spatial distribution of populations within a given geography. And there are other measures like the Exposure Index, which measures how likely it is that two groups will make contact with each other in the same geography. There is no single measure that will prove or not prove segregation, but the measures can work together to explain how a community is comprised.

I read a lot of research on census demographics and tried to mirror my categories to existing literature on the topic. Thus, I chose the six race categories included in this project based on existing research about race and segregation that was commissioned by the Census Bureau, and chose the Multigroup Entropy Index because it allowed me to compare multiple racial groups in a single analysis.

I decided to compare the makeup of each census block group to the racial makeup of its surrounding county.

Then, my colleague Armand Emamdjomeh and I spent months working on the pipeline that powered the data analysis. In the past, I’ve seen a lot of census demographic research done in tools like Python, R or SPSS but I was curious if I could do this work using JavaScript. I found JavaScript and the node.js ecosystem to provide a rich set of tools to work with data and then display it on the web.

One challenge was that I had to write several of my analysis functions by hand, but in return I was able to understand every step of my analysis and use the same functions on the web. Mapbox and d3.js both have very powerful and mature tools for working with geospatial data that I leveraged at each stage of my analysis.

About two months before the story was published, we went back and forth on the story design and layout. An early version of this project implemented the scrollytelling approach, where the map took over the entire screen and the text scrolled over the map.

While this approach is well established and used heavily by my team at the Post, it prevented us from including the beautiful static graphics we generated in a holistic way. In the end, we opted for a traditional story layout that explored the history of segregation and housing discrimination in the United States, complete with case studies on three cities, and then included the full, historical interactive map at the bottom.1

The story is the most read project I have ever published as a journalist. I think letting readers explore the data after the story added a layer of personalization that allowed readers to situate themselves in the narrative. Data journalism allows us to tell stories that go beyond words, beyond ideas. We can put the reader directly into the story and let them tell their own.

Footnotes

1.www.washingtonpost.com/graphics/2018/national/segregation-us-cities

Written by Christoph Koettl

In the early summer of 2014, Amnesty International received a video depicting Nigerian soldiers slitting the throats of suspected Boko Haram supporters, and then dumping them into a mass grave. The video, which circulated widely in the region and on YouTube, implicated Nigerian soldiers in a war crime. However, in order to draw that conclusion, we undertook an extensive investigation involving video analysis and field research. This resulted in the publication of Amnesty International’s (AI) findings of this incident.

This incident is a powerful example of how user-generated content can contribute to in-depth investigations. It also demonstrates the importance of digging deeper and going beyond the basic facts gathered from standard UGC verification. This is particularly important for human rights investigations. UGC not only aids in determining the place and time of a violation; it can also help with identifying responsible individuals or units (linkage evidence) that can establish command responsibility, or with providing crucial crime base evidence that proves the commission of a crime.

While there are differences between human rights and war crimes investigations and journalistic reporting, there is also immense overlap, both in regards to the verification tools used and in terms of the benefits of relying on UGC. In fact, the British media outlet Channel 4 conducted an investigation into the conflict in northeastern Nigeria that was largely built on the same UGC footage.

Principles of human rights investigations

While a lot of UGC might have immense news value, human rights groups are of course primarily interested in its probative value. In a human rights investigation, we compare all facts gathered with relevant human rights norms and laws (such as human rights and humanitarian, refugee and criminal law) to make determinations of violations or abuses. Consequently, a single analyst who looks at UGC, such as myself, must be part of a team comprising relevant country, policy and legal experts.

Our ultimate goal is to achieve a positive human rights impact, such as when our work contributes to establishing an international inquiry, or the indictment of a suspected perpetrator. Today we are achieving the best results when combining a variety of evidence, such as testimony, official documents, satellite imagery and UGC.

This requires the close collaboration of researchers who possess country expertise, trusted contacts on the ground, and highly specialized analysts who do not focus on a specific region or country, but are able to provide analysis based on satellite imagery or UGC.

In some instances, one piece of evidence does not corroborate some of the information gathered during the investigation, such as when satellite imagery does not support eyewitness claims of a large mass grave. We then exercise caution and hold back on making statements of fact or determinations of violations.

This close collaboration among a range of experts becomes even more relevant when going beyond war crime investigations, which can be based on a single incident caught on camera. Crimes against humanity, for example, are characterized by a systematic and widespread nature that is part of a state or organizational policy. Research solely based on UGC will hardly be able to make such a complex (legal) determination. It usually provides only a snapshot of a specific incident. However, it can still play a crucial role in the investigation, as the following example will show.

War crimes on camera

In 2014, AI reviewed dozens of videos and images stemming from the escalating conflict in northeastern Nigeria. Human rights groups and news organizations have extensively documented abuses by Boko Haram in the country. But this content proved especially interesting, as the majority of it depicts violations by Nigerian armed forces and the state-sponsored militia Civilian Joint Task Force (CJTF).

The most relevant content related to events March 14, 2014, when Boko Haram attacked the Giwa military barracks in Maiduguri, the state capital of Borno state. The attack was captured on camera and shared on YouTube by Boko Haram for propaganda purposes. It resulted in the escape of several hundred detainees. The response by authorities can only be described as shocking: Within hours, Nigerian armed forces and the CJTF extra-judicially executed more than 600 people, mostly recaptured detainees, often in plain sight, and often on camera.

Thorough research over several months allowed us to connect different video and photographs to paint a disturbing picture of the behavior of Nigerian armed forces. For example, one grainy cellphone video showed a soldier dragging an unarmed man into the middle of a street and executing him, next to a pile of corpses.

We first performed standard content analysis. This involved extracting the specifications of the road and street lamps, buildings and vegetation, as well as details related to the people seen in the video, such as clothes and military equipment. Reviewing the video frame by frame greatly aided with this process. The geographic features were then compared to satellite images of the area on Google Earth.

Based on this work, it was possible to pinpoint the likely location within Maiduguri, a large city of around a million people.

Several months later, additional photographs, both open source and directly collected from local sources, were used to paint a more comprehensive and even more worrisome picture of the incident. For example, at least two of the victims had their hands tied behind their backs. It is noteworthy that several photographs in our possession were actually geotagged. We discovered this by using a EXIf reader to examine the metadata in the photo. This location data proved a perfect match to the street corner we identified as part of the content analysis of the initial video.

Other videos from the same day documented an even more gruesome scene, which suggested another war crime. They show the killing of several unarmed men, as detailed earlier in this chapter. The videos were a textbook example of how UGC can be a powerful tool in longterm investigations when combined with traditional investigative methods.

We slowed the video to perform a content analysis in order to identify distinctive markings on the soldiers and victims, or anything that could indicate location, time or date. This revealed two important details: a soldier wearing a black flak jacket stating “Borno State. Operation Flush,” the name of the military operation in northeastern Nigeria; and, for a split second, an ID number on a rifle (“81BN/SP/407”) became visible. No distinctive geographic features were visible that could be used to identify the exact location.

AI subsequently interviewed several military sources who independently confirmed the incident, including the date and general location outside of Maiduguri. An AI researcher was also able to secure the actual video files while on a field mission to the area. This allowed us to conduct metadata analysis that is often not possible with online content, since social media sites regularly modify or remove metadata during the upload process.

The data corroborated that the footage had been created March 14, 2014. Obtaining the original files is often possible only through well-established local contacts and networks, who might share content in person or via email (ideally encrypted). Savvy news desk researchers and journalists who might be inclined to contact local sources via Twitter or other public platform should consider the risk implications for asking for such sensitive footage from contacts in insecure environments.

In this case, two sources stated that the perpetrators may be part of the 81 Battalion, which operates in Borno state, and that the rifle ID number refers to a “Support Company” of that battalion. Most important, several sources, who had to remain anonymous, separately stated that this specific rifle had not been reported stolen, disqualifying the predictable response by Nigerian authorities that the soldiers were actually impostors using stolen equipment.

After an initial public statement about the most dramatic footage, AI continued its investigation for several months, bringing together traditional research, such as testimony, with satellite imagery and the video footage and photographs detailed above. This UGC supported the overall conclusion of the investigation that both Boko Haram and Nigerian armed forces were also implicated in crimes against humanity. These findings can have serious implications, as the violations detailed are crimes under international law, and are therefore subject to universal jurisdiction and fall under the jurisdiction of the International Criminal Court.

Written by Steve Buttry

In 1996, I did a project on an American high school girls basketball team that had won the Iowa state championship 25 years earlier. I interviewed all 12 members of the Farragut team, as well as the star and coach of Mediapolis, the team Farragut beat for the championship.

I asked them all how Farragut won the game. They gave different, often vivid, accounts of the same story: Mediapolis star Barb Wischmeier, who was 6 feet tall, scored easily on the shorter Farragut girls early in the game, and Mediapolis took the lead.

The Farragut coach sent Tanya Bopp, who was barely 5 feet, into the game to guard Wischmeier. Bopp drew several charging fouls (some remembered specifically that it was three or four fouls) on the larger girl, who became flustered and less aggressive. Farragut came back to win the game.

I didn’t question these consistent memories in my reporting, but learned almost by accident that they were exaggerated. One of the girls loaned me a video of the game. I watched the whole game, looking for details that would help my story. I wasn’t challenging anyone’s memory, but when I finished the tape, I thought I must have missed something. So I watched it again.

Tiny Tanya Bopp drew only one foul on the larger girl. It did fluster the Mediapolis star and was the turning point of the game, but it happened only once. All those firsthand accounts I had heard were inaccurate, fueled by the emotions (joy or anguish) of an important moment in their lives, and shaped by a legend that grew from the game.

The legend - and the opportunity to honor it by debunking it - gave me a great narrative thread for my article but also taught me a lesson in verification: Don’t trust even honest witnesses. Seek documentation.

Legends are fine, and even fun, for athletes and fans reliving the glory days of a legendary sports team. But journalists, activists or human rights workers must deal with the truth and must be committed to finding and telling the truth, especially in an emergency situation.

Whether we’re assembling the tale of a natural disaster, a breaking news story or a bit of popular lore, storytellers must remember that we hear the product of faulty memory or limited perspective. If telling the truth is our goal, verification must be our standard.

We need to look and listen earnestly to the stories of our sources, watching for opportunities to verify. Does the source have a (new or old) video, photograph, letter or document that can offer verification or detail, or perhaps correct a foggy memory? And when we’re supplied with this material, especially in emergency situations where time is tight, we need to investigate it and apply the fundamentals of verification.

Regardless of the moment and your role in it, the principles of verification are timeless and can be applied to any situation, be it breaking news, a natural disaster or the retelling of a apocryphal tale from a quarter century earlier.

The Essence of Verification

One of journalism’s most treasured clichés, spouted by seasoned editors who ruthlessly slash other clichés from stories, is: “If your mother says she loves you, check it out.”

But the cliché doesn’t tell the journalist, or humanitarian professional, how to check it out. Verification is the essence of journalism, but it also illustrates the difficulty of journalism and the need for high standards: The path to verification can vary with each fact.

So this handbook won’t present journalists, human rights workers and other emergency responders with one-size-fits-all simple steps to verification, but with strategies to check it out - whatever “it” is, and whatever motivation or role you have.

The question at the heart of verification is: “How do you know that?”

Reporters need to ask this question of their sources; editors need to ask it of reporters. Reporters, editors, producers and human rights workers need to ask the question in the third person about sources they can’t ask directly: How do they know that?

Newsroom coach Rosalie Stemer adds a second question that illustrates the multilayered process of verification and the ethic of persistence and resourcefulness that verification demands: How else do you know that?

As we question sources and material, and as colleagues question us, we need to seek multiple sources of verification, multiple paths to the truth. (Or, to finding holes in the data or story before we act on it.)

Verification employs a mix of three factors:

1. A person’s resourcefulness, persistence, skepticism and skill
2. Sources’ knowledge, reliability and honesty, and the number, variety and reliability of sources you can find and persuade to talk
3. Documentation

Technology has changed how we apply all three factors: The 24/7 news cycle and rise of social media and user-generated content require us to gather and report as events unfold, making swift decisions about whether information has been sufficiently verified; digital tools give us new ways to find and reach sources; databases and ubiquitous cellphones with cameras give us massive amounts of documentation to seek and assess. Successful verification results from effective use of technology, as well as from commitment to timeless standards of accuracy.

The need for verification starts with the simple fact that many of our information sources are wrong. They may be lying maliciously or innocently passing along misinformation. They may have faulty memories or lack context or understanding. They may be in harm’s way and unable to provide everything they know, or unable to see the full picture of events as they unfold.

Our job is not to parrot sources and the material they provide, but to challenge them, triangulate what they provide with other credible sources and verify what is true, weeding from our work (before we publish, map or broadcast) what is false or not adequately verified.

Each of the many verification paths that we might take has its flaws: In many cases, and especially in emergency situations, we are increasingly presented with an abundance of official sources and can find firsthand sources, the people who actually saw - or even participated - in the events in question. But those accounts can be flawed.

West Virginia Gov. Joe Manchin told reporters in 2006 that 12 of 13 miners trapped underground had been rescued from the Sago mine. What reporter wouldn’t run with that story?

But the governor was wrong. Twelve of the miners died; only one was rescued. The governor relied on second- and thirdhand accounts, and was not challenged on how he knew the miners were alive. We need to question seemingly authoritative sources as aggressively as we challenge any source.

New Tools

Documentation has changed with technology. The video that helped me debunk the legend in 1996 wouldn’t have been available from one of the team members if I’d tried doing that story 15 years earlier (though I still could have watched it by going to the archives of the TV station). And in the years since I used that video for verification, the availability of cellphones and security cameras has increased the amount and importance of video documentation. But the ease of digital video editing raises the importance of skepticism. And, of course, any video catches only part of the story.

Technology has also changed how we find and deal with sources and information. As participants and witnesses to news events share their accounts in words, photos and videos on social media and blogs, journalists can more quickly find and connect with people who saw news unfold both by using digital search tools and other technologies, and by crowdsourcing.

We can use new tools most effectively by employing them with those old questions: How do they know that? How else do they know that?

That old cliché about checking out Mom’s love? I verified the source (the old Chicago City News Bureau) from multiple online sources: the Chicago Tribune, AJR and The New York Times. Even there, though, legend complicates verification. A 1999 Baltimore Sun article by Michael Pakenham said legend attributes the admonition to the bureau’s longtime night city editor, Arnold Dornfeld (as three of the articles linked above do), but “Dornie said it was another longtime editor there, Ed Eulenberg, who actually said it first.”

What is data journalism? I could answer, simply, that it is journalism done with data. But that doesn’t help much.

Both ‘data’ and ‘journalism’ are troublesome terms. Some people think of ‘data’ as any collection of numbers, most likely gathered on a spreadsheet. 20 years ago, that was pretty much the only sort of data that journalists dealt with. But we live in a digital world now, a world in which almost anything can be — and almost everything is — described with numbers.

Your career history, 300,000 confidential documents, who knows who in your circle of friends can all be (and are) described with just two numbers: zeroes, and ones. Photos, video and audio are all described with the same two numbers: zeroes and ones. Murders, disease, political votes, corruption and lies: zeroes and ones.

What makes data journalism different to the rest of journalism? Perhaps it is the new possibilities that open up when you combine the traditional ‘nose for news’ and ability to tell a compelling story, with the sheer scale and range of digital information now available.

And those possibilities can come at any stage of the journalist’s process: using programming to automate the process of gathering and combining information from local government, police, and other civic sources, as Adrian Holovaty did with ChicagoCrime and then EveryBlock.

Or using software to find connections between hundreds of thousands of documents, as The Telegraph did with MPs' expenses.

Data journalism can help a journalist tell a complex story through engaging infographics. Hans Rosling’s spectacular talks on visualizing world poverty with Gapminder, for example, have attracted millions of views across the world. And David McCandless’s popular work in distilling big numbers — such as putting public spending into context, or the pollution generated and prevented by the Icelandic volcano — shows the importance of clear design at Information is Beautiful.

Or it can help explain how a story relates to an individual, as the BBC and the Financial Times now routinely do with their budget interactives (where you can find out how the budget affects you, rather than ‘Joe Public’). And it can open up the news gathering process itself, as The Guardian do so successfully in sharing data, context, and questions with their Datablog.

Data can be the source of data journalism, or it can be the tool with which the story is told — or it can be both. Like any source, it should be treated with scepticism; and like any tool, we should be conscious of how it can shape and restrict the stories that are created with it.

Written by Fergus Bell

User-generated content (UGC) is taking an increasingly prominent role in daily news coverage, with audiences choosing to share their stories and experiences through the content they create themselves. Our treatment of the people who share this compelling content has a direct impact on the way that we, and other organizations, can work with them in the future.

It is essential to determine what ethical standards will work for you and your audience, and what actions will allow you to establish and preserve a relationship with them. Our approach must be ethical so that it can be sustainable.

Individuals contribute to news coverage in two typical ways. In one, journalists can invite and encourage people to participate in programming and reporting. This type of contributor will often be loyal, create content in line with the organization’s style, and will be conscientious with any contributions.

The second type of contributor is the “accidental journalist.” This could be an eyewitness to an event, or someone sharing details that will aid your investigation, even if that person may not be doing so with the idea of assisting journalists. These types of contributor often have little or no idea that what they have to offer, or are inadvertently already offering, may be of value or interest to journalists. This is especially true in the context of investigative reporting.

This chapter highlights some key questions and approaches when applying ethics and standards to newsgathering from social media, and when working with user-generated content.

Entering private communities

Private communities can be extremely fruitful for generating investigative leads. Obvious examples of private communities are blogs, subreddits and Facebook groups. A less-obvious private community might be when an individual uses a YouTube page to share videos with friends and family. It’s a public account, but the user assumes a level of privacy because the material is being shared with specific people. The key takeaway here is to consider how the content creator sees their activity, rather than how you see it. This will help you apply the most sensitive and the most ethical approach.

The main issue is likely to be how you identify yourself to and within that community. Within your organization, you need to consider two questions about how transparent you should be.

1. When is anonymity acceptable? — Users on platforms such as Reddit and 4Chan are mostly anonymous, and it might be acceptable to start interactions without first identifying yourself as a journalist. However, if you are more than just conversation-watching, there will likely be a time when it’s appropriate to identify yourself and your profession. Reddit recently issued guidance on how to approach its community when working on stories. These should be consulted when utilizing that platform.

2. When is anonymity unlikely to be an option? — Networks such as Facebook and Twitter are often more useful for breaking news because people are more likely to use real names and identities. In this kind of environment, anonymity as a journalist is less of an option. Again, if you are just watching rather than engaging with individuals, then being open and honest about who you are is often going to be the best way forward.

There are always going to be exceptions to the rule. This is also the case when it comes to deciding when it’s acceptable for journalists to go undercover in the real world. Working out your policy before you need it is always going to yield the best results. You can then act with the confidence that your approach has been properly thought through.

Securing Permission

Seeking permission to use content from creators of UGC helps establish and maintain the reputation of your organization as one that gives fair treatment. Securing permission will also help you ensure you are using content from an original source. This may save you legal headaches in the long run. All of the principal social platforms have simple methods for communicating quickly and directly with users.

Communication with individuals is, of course, an important part of any verification process. This means the act of asking for permission also opens up a potential source of additional information or even content that you otherwise wouldn’t have had.

The question of payment for content is a separate issue that your organization needs to determine for itself. But it’s clear that securing permission and then crediting is the new currency for user-generated content. Claire Wardle covers this in the next chapter.

Contributor management and safety

Audience contributions/assignments

If you are gathering content from your audience through requests or assignments, then there are several ethical issues to take into account. At the top of the list is your responsibility to keep them safe.

When devising standards in this area, you should discuss the following issues:

• Does an assignment put someone at risk?
  

• Could an individual get too close to a dangerous event or to people who may cause them harm?

• What is your responsibility to a person who is harmed while carrying out an assignment set by you?
  

• How will you identify this person in the publication or broadcast?
  

• What impact does an assignment have on the honesty/authenticity of the content being produced versus something that was created unprompted?

Discovered content

The above issues also apply to those people whose contributions you’ve discovered, as opposed to having them sent to you. However, in the case of accidental journalists, there are additional questions you need to ask within your organization. These help establish your policy for communicating with them and for using their content:

• Does the person realize how they might be affected by sharing this content with the media?
  

• Do you think the owner/uploader knew that their content was discoverable by organizations like yours? Do you think they intended it for their personal network of friends and family?
  

• For something that is particularly newsworthy, how can you seek permission or contact with them without bombarding them as an industry?
  

• How can you sensitively communicate with individuals who have something newsworthy but are perhaps in a situation which has caused them distress, or loss?
  

• Does the publication or broadcast of their content identify their location or any personal information that might cause them to be harmed or otherwise affected?

Charting an ethical course for the future

The Online News Association has several initiatives to address many of the issues raised in this chapter. The aim is to create resources that will allow journalists at all types of news organizations to chart an ethical course for the future.

The ONA’s DIY ethics code project allows newsrooms to devise a personalized code of ethics. The ONA’s UGC working group was established to bring leaders together from across the journalism community to freely discuss challenges and possible solutions to the ethical issues raised by the increased use of social newsgathering and UGC.

The group is focusing on three specific areas:

• Can the industry agree on an ethical charter for UGC?
  

• Can we work with the audience to understand their needs, frustrations and fears?

• How can we further protect our own journalists working with UGC?

Those interested in becoming a member of this working group can join our Google+ community.

Written by Anthony de Rosa

The medium by which we’re gathering information may change, but the principles of verification always apply. Challenging what you see and hear, seeking out and verifying the source, and talking to official and primary sources remain the best methods for accurate reporting.

At Circa, we track breaking news from all over the world - but we publish only what we can confirm. That requires that we use social media to monitor breaking news as it happens so we can apply verification

Remember that the information on social media should be treated the same as any other source: with extreme skepticism.

For the most part, I view the information the same way I would something I heard over a police scanner. I take in a lot and I put back out very little. I use the information as a lead to follow in a more traditional way. I make phone calls, send emails and contact primary sources who can confirm what I’m hearing and seeing (or not).

In the case of the 2013 shooting at the Los Angeles airport, for example, we observed reports from the airport coming from eyewitnesses and contacted LAPD, the LA FBI field office and the LA county coroner. If we couldn’t independently verify what we saw and heard, we held it until we could.

Even in cases where major news organizations were reporting information, we held back until we could confirm with primary sources. Often these organizations cite unnamed law enforcement sources, and as we’ve seen with the Boston Marathon bombing, the Navy Yard shooting, the Newtown shooting and other situations, anonymous law enforcement sourcing is often unreliable.

Using TweetDeck to monitor updates

If social media is a police scanner, TweetDeck is your radio. There are a few ways you can create a dashboard for yourself to monitor the flow of updates.

I build Twitter lists ahead of time for specific uses. My list topics include law enforcement for major cities, reliable local reporters and news organizations for major cities, and specialized reporters. I can plug these lists into columns on TweetDeck and run searches against them, or simply leave them up as a monitoring feed.

Small plane lands in the Bronx

Here’s how I used searches on TweetDeck during the January 2014 emergency landing of a small plane on a Bronx expressway to unearth breaking news reports and to triangulate and verify what I saw.

I noticed several tweets appear in my main timeline mentioning a plane landing on the Ma- jor Deegan Expressway in the Bronx section of New York, which is not a normal occurrence.

The plane landed around 3:30 p.m. local time in New York. (The tweet is dated in Pacific Standard Time.) This was one of the first tweets to report the landing. I follow a couple of NYC area accounts like, which act as a sort of police scanner for what’s going on in the area. I won’t report it until I can back it up, but it’s useful to have as a potential alert to dig deeper.

After seeing the initial reports, I proceeded to run a search on TweetDeck using its ability to show tweets that only have images or video. I used the search terms “small plane” and “Bronx.”

The above results showed that credible local news sources were reporting the plane landing, and they had images. I also found additional information and images from a wider search of all tweets that used a location filter (within 5 miles of New York City) and the keywords “small plane” and “bronx”:

I also searched within my specialized list of verified accounts belonging to New York State and City agencies, and used the location filter again. These credible sources (below) helped confirm the event.

At this point I contacted the public information office for the FDNY to confirm what I saw and ask for any other details they might have. I was told there were three people on board, two passengers and a pilot. We were later told the make/model of the plane, the name of the person the plane was registered to, and the hospital the pilot and pas- sengers were taken to. Social media led us to the event - but we had to track the details down the old-fashioned way.

Feeling we had properly run down enough credible information to get started, we filed our story (see below). The Circa app offers readers an option to “follow” a story and receive push updates as more information is added. Our process is to get a story up as soon as possible with verified reports and continue to push out updates. TweetDeck allows us to get a jump on a developing story and seek out reliable people (law enforcement, primary sources) we can contact to confirm the validity of social media updates. In some cases we contact the person who sent the information to Twitter and try to determine if they’re reliable.

Building a body of evidence

The information you’re seeing on social media should be the first step toward trying to verify what actually occurred, rather than the final word.

The key is to observe as much as you can, take in information and compare it to other content and information to build up a body of evidence. Find ways to corroborate what you find by directly contacting and verifying the people who are connected to the content you find.

As I said, treat social media as a police scanner.

Dr. Joan Donovan is the Research Director at Harvard Kennedy’s Shorenstein Center on Media, Politics and Public Policy

In an age where a handful of powerful global tech platforms have disrupted the traditional means by which society is informed, media manipulation and disinformation campaigns now challenge all political and social institutions. Hoaxes and fabrications are propagated by a mixed group of political operatives, brands, social movements and unaffiliated “trolls” who have developed and refined new techniques to influence public conversation, wreaking havoc on a local, national and global scale. There’s widespread agreement that media manipulation and disinformation are important problems facing society. But defining, detecting, documenting and debunking disinformation and media manipulation remains difficult, especially as attacks cross professional sectors such as journalism, law and technology. Therefore, understanding media manipulation as a patterned activity is an essential first step in working to investigate, expose and mitigate them.

Defining media manipulation and disinformation

To define media manipulation, we first split the term in two parts. In its most general form, media is an artifact of communication. Examples include text, images, audio and video in material and digital mediums. When studying media, any relic can be used as recorded evidence of an event. Crucially, media is created by individuals for the purpose of communicating. In this way, media conveys some meaning across individuals, but interpreting that meaning is always relational and situated within a context of distribution.

To claim media is manipulated is to go beyond simply saying that media is fashioned by individuals to transmit some intended meaning. The Merriam-Webster dictionary defines manipulation as “to change by artful or unfair means so as to serve one’s purpose.” While it can sometimes be difficult to know the exact purpose a single artifact was created to serve, investigators can determine the who, what, where and how of its communication to help determine if manipulative tactics were used as part of the distribution process. Manipulation tactics can include cloaking one’s identity or the source of the artifact, editing to conceal or change the meaning or context of an artifact, and tricking algorithms by using artificial coordination, such as bots or spamming tools.

In this context, disinformation is a subgenre of media manipulation, and refers to the creation and distribution of intentionally false information for political ends. Technologists, experts, academics, journalists and policymakers must agree on the distinctive category of disinformation because efforts to fight against disinformation require the cooperation of these groups.

For our part, the Technology and Social Change research team (TaSC) at Harvard Kennedy School’s Shorenstein Center is using a case study approach to map the life cycle of media manipulation campaigns. This methodological approach seeks to analyze the order, scale and scope of manipulation campaigns by following media artifacts through space and time, drawing together multiple relationships to sort through the tangled mess. As part of this work, we’ve developed an overview of the life cycle of a media manipulation campaign, which is useful for journalists as they attempt to identify, track and expose media manipulation and disinformation.

The life cycle has five points of action, where the tactics of media manipulators can be documented using qualitative and quantitative methods. Note that most manipulation campaigns are not “discovered” in this order. Instead when researching, look for any one of these points of action and then trace the campaign backward and forward through the life cycle.

Case study: ‘Blow the Whistle’

Let’s examine the social media activity around the whistleblower complaint made about the activity of President Donald Trump related to Ukraine to see how a media manipulation campaign unfolds, and how ethical action by journalists and platforms early in the life cycle can help thwart manipulation efforts.

Planning and Seeding (Stages 1 & 2) — In the conspiracy theory media ecosystem, the whistleblower’s identity is already known and his name is circulating on blogs, Twitter, Facebook, YouTube videos and discussion forums. Importantly, unique names can substitute for keywords and hashtags, which function as discrete searchable data points. There was a concerted push to spread the alleged name and the person’s photo. Yet, the name seems to be locked in this online media echo chamber of right-wing and conspiracy accounts and entities. Even with this coordinated effort by conspiracy-themed influencers to push the alleged whistleblower’s name into the mainstream, they were not able to break out of their own filter bubbles. Why is that?

Responses by journalists, activists etc. (Stage 3) — In contrast, leftist and centrist media did not print the name of the alleged whistleblower or amplify claims that he was outed. Mainstream media outlets refrained from calling attention to the circulation of this person’s name in the social media ecosystem, even though it’s a newsworthy story for reporters on the tech and politics beat. Those that did cover it often emphasized how the act of circulating this name was an attempt to manipulate the discussion around the whistleblower’s complaint, and avoided spreading the name. This is due in large part to the ethics of journalism, where reporters have a special duty to protect the anonymity of sources, which extends to whistleblowers.

Changes to information ecosystem (Stage 4) — While mainstream journalists were omitting his name, the alleged name of the whistleblower, “Eric Ciaramella,” is a unique keyword. This meant that people who searched for it could pull up a wide variety of content rooted in the conspiracy-influenced point of view. In addition to ethical journalists effectively turning down a story that could attract significant traffic, each platform company began actively moderating content that used the alleged whistleblower’s name as a keyword. YouTube and Facebook removed content that used his name, while Twitter prevented his name from trending. Google’s search did allow for his name to be queried and returned thousands of links to conspiracy blogs.

Adjustments by manipulators (Stage 5) — Manipulators were aggravated by these attempts to prevent the spread of misinformation and changed their tactics. Instead of pushing content with the alleged whistleblower's name, manipulators began circulating images of a different white man (with glasses and a beard) that resembled the image they previously circulated with his name. These new images were coupled with a “deep state” conspiracy narrative that the whistleblower was a friend of establishment Democrats, and therefore had partisan motives. However, this was an image of Alexander Soros, the son of billionaire investor and philanthropist George Soros, a frequent target of conspiracies.

When that failed to generate media attention, President Trump’s Twitter account, @RealDonaldTrump, retweeted an article giving the alleged whistleblower’s name, emphasizing that “The CIA whistleblower is not a real whistleblower!” to his 68 million followers. The original tweet came from @TrumpWarRoom, which is his campaign’s official and verified account. A cascade of media coverage followed, including many major mainstream outlets, all of which took pains to remove or cover the alleged whistleblower’s name. Many people called on social media for the whistleblower to testify in the Senate impeachment hearings, where his name was invoked alongside other important potential witnesses, broadening the possibility that others will stumble on it when searching for other names. And thus begins a new cycle of media manipulation.

Queries for the name of the whistleblower are on the rise and conspiracies abound on blogs about his personal and professional motivations for informing on Trump’s activities. Journalists reporting on these tweets oscillate between discussions of witness intimidation, citing that an act like this can deter future whistleblowers, while also tipping into lurid curiosity by reporting on the gossip surrounding Trump’s motive for outing the alleged whistleblower. As such, it is laudable that some media organizations are trying to hold elites to account, but the task is impossible without the platform companies’ addressing how their products have become useful political tools for media manipulation and spreading disinformation.

Documenting the life cycle

Media manipulators attempted to “trade up the chain” by seeding a name and photos on social media in order to eventually cause large, legitimate media to amplify it, where platforms would allow it to trend and become easily discoverable. But decisions and actions by platforms and journalists meant the attempt to push the alleged identity of the whistleblower into mainstream consciousness largely failed until a newsworthy figure pushed the issue. While many media organizations strive to abide by ethical guidelines, social media has become a weapon of the already powerful to set media agendas and drive dangerous conspiracies.

Generally speaking though, this case study is a significant improvement over prior efforts to stop the spread of disinformation, where journalists amplified disinformation campaigns as they tried to debunk them, and platform companies felt no duty to provide accurate information to audiences. This overall shift is promising, but accountability for elites is still lacking. For journalists and researchers alike, the stakes of detecting, documenting and debunking media manipulation campaigns are high. In this hyperpartisan moment, any claim to name a disinformation campaign may also bring hordes of trolls and unwanted attention. Grappling with the content and the context of disinformation requires us all to forensically document with rigor how campaigns start, change and end. And to recognize that every perceived ending of a campaign may very well be a new beginning.

Abstract

How we used data about trees to create memories, promote transparency and include citizens in storytelling in Bogotá, Colombia.

Keywords: data journalism, citizenship, transparency, open government, multimodal storytelling, trees

Bogotá holds almost 16% of the population of Colombia in just 1.775 km².

You get the idea, it’s crowded, it’s furious. But it’s also a green city, surrounded by mountains and many different trees planted.

Most of the times, trees go unnoticed by its citizens in the midst of their daily life. Or at least that’s what happened to the members of our data team except for one of our coders, who loves trees and can’t walk down the street without noticing them. She knows all the species and the facts about them. Her love for nature in the midst of the chaos of the city is what got us thinking: has anybody, ever, talked about the trees that are planted all over town?

And that simple question was the catalyst for so many others:

-What do we know about them?

-Who is in charge of taking care of them?

-Are they really useful to clean the city’s pollution?

-Do we need more trees in the city?

-Is it true that only the rich neighborhoods have tall trees?

-Are there any historical trees in town?

We began our investigation aiming to do two different things: Firstly, to connect the citizens with the green giants they see everyday; and secondly, to understand the reality of the city’s tree planting and conservation plans.1

To do so, we analyzed the urban census of tree planting in Bogotá that the Botanical Garden conducted in 2007, the only set of information available, and which is updated every month.

The Botanical Garden refused to give us the full data even after we submitted multiple freedom of information requests filled with legal arguments. Their position was simple: The data was already available in their DataViz portal.

Our argument: You can only download 10,000 entries and the database is made up of 1.2 million entries. It’s public data, just give it to us! Their answer: We won’t give it to you but we will improve our app so you can download 50,000 entries.

Our solution? Reach out to other organizations that had helped the Botanical Garden collect the data. One of those entities was Ideca, which collects all the information related to the city’s cadastre.

They gave us the whole data set in no time. We, obviously, decided to publish it so that everyone can access it (call it our little revenge against opacity). The Botanical Garden realized this and stopped any further conversation with us, and we decided not to continue a legal battle.

In addition, we included public data from the Mayor’s Office of Bogotá and the National Census, to cross-reference information that we could analyze in relation to trees.

Finally, we conducted interviews with environmental experts and forestry engineers that allowed us to understand the challenges the city faces. They had done so much work and so many investigations analyzing not only the reality of tree planting schemes, but also the history behind the trees in the city. And most of this work was largely unnoticed by authorities, journalists and many others.

The final product was an eight-piece data project that showed the reality of the tree planting plans of the city. It mapped every single tree—with information about its height, species and benefits for the city—debunked many myths around tree planting, and told the stories of some of the city’s historical trees.

We used Leaflet and SoundCloud for the interactive elements. The design was implemented by our talented group of coders. We also used StoryMapJS to allow users to explore the historic trees of the city.

We decided how and which pieces were important for the story after researching many other similar projects and then partnered with a designer to create a good user experience. It was our first big data project and a lot of it involved trial and error as well as exploration.

More importantly, we involved citizens by inviting them to help us build a collaborative tree catalogue and to share their own stories about the trees we had mapped.

We did so through social media, inviting them to add information about tree species to a spreadsheet. Bogotá’s residents continue to help us enrich the catalogue to this day.

In addition, we shared a WhatsApp number where people could send voice notes with their stories about trees. We received almost a hundred voice messages from people telling stories of trees where they had their first kiss, that taught them how to climb, that protected them from thieves or that were missed because they were cut down.

We decided to include these audio files as an additional layer in the visualization app, so users could also get to know the city’s trees through people’s stories.

The main article and visual was then republished by a national newspaper (both in print and online), and shared by local authorities and many residents who wanted to tell their stories and transform the relationship that other residents have with their environment.

So far, people have used the map to investigate the city’s nature and to support their own research on the city’s trees.

For our organization, this has been one of the most challenging projects we have ever developed. But it is also one of the most valuable, because it shows how data journalism can be about more than just numbers: It can also play a role in creating, collecting and sharing culture and memories, help people notice things about the places they live (beyond graphs and charts), and multiply and change the relations between people, plants and stories in urban spaces.

Footnotes

1. especiales.datasketch.co/arboles-bogota

Journalism is under siege. In the past we, as an industry, relied on being the only ones operating a technology to multiply and distribute what had happened over night. The printing press served as a gateway, if anybody wanted reach the people of a city or region the next morning, they would turn to newspapers. This is over.

Today news stories are flowing in as they happen, from multiple sources, eye-witnesses, blogs and what has happened is filtered through a vast network of social connections, being ranked, commented and more often than not: ignored.

This is why data journalism is so important. Gathering, filtering and visualizing what is happening beyond what the eye can see has a growing value. The orange juice you drink in the morning, the coffee you brew — in today’s global economy there are invisible connections between these products, other people and you. The language of this network is data: little points of information that are often not relevant in a single instance, but massively important when viewed from the right angle.

Right now, a few pioneering journalists already demonstrate how data can be used to create deeper insights into what is happening around us and how it might affect us.

Data analysis can reveal “a story’s shape” (Sarah Cohen), or provides us with a “new camera” (David McCandless). Using data the job of journalists shifts its main focus from being the first ones to report to being the ones telling us what a certain development might actually mean. The range of topics can be far and wide. The next financial crisis that is in the making. The economics behind the products we use. The misuse of funds or political blunders, presented in a compelling data visualization that leaves little room to argue with it.

This is why journalists should see data as an opportunity. They can, for example, reveal how some abstract threat such as unemployment affects people based on their age, gender, education. Using data transforms something abstract into something everyone can understand and relate to.

They can create personalized calculators to help people to make decisions, be this buying a car, a house, deciding on an education or professional path in life or doing a hard check on costs to keep out of debt.

They can analyze the dynamics of a complex situation like riots or political debates, show the fallacies and help everyone to see possible solutions to complex problems.

Becoming knowledgeable in searching, cleaning, and visualizing data is transformative for the profession of information gathering, too. Journalists who master this will experience that building articles on facts and insights is a relief. Less guessing, less looking for quotes — instead, a journalist can build a strong position supported by data and this can affect the role of journalism greatly.

Additionally, getting into data journalism offers a future perspective. Today, when newsrooms cut down, most journalists hope to switch to public relations. Data journalists or data scientists though are already a sought-after group of employees, not only in the media. Companies and institutions around the world are looking for “sensemakers” and professionals, who know how to dig through data and transform it into something tangible.

There is a promise in data and this is what excites newsrooms, making them look for a new type of reporter. For freelancers proficiency with data provides a route to new offerings and stable pay, too. Look at it this way: instead of hiring journalists to quickly fill pages and websites with low value content the use of data could create demand for interactive packages, where spending a week on solving one question is the only way to do it. This is a welcome change in many parts of the media.

There is one barrier keeping journalists from using this potential: training in order to learn how to work with data through all the steps from a first question to a big data-driven scoop.

Working with data is like stepping into vast, unknown territory. At first look, raw data is puzzling to the eyes and to the mind. Data as such is unwieldy. It is quite hard to shape it correctly for visualization. It needs experienced journalists, who have the stamina to look at often confusing, often boring raw data and “see” the hidden stories in there.

Ten years ago, a huge earthquake in the Indian Ocean unleashed a devastating tsunami across the region. At first, there were no pictures of the wave; it took a couple of days for the first images to surface. And when they did appear, most were shaky footage, captured mostly by tourists pressing record on their camcorders as they ran to safety. None of them expected their home videos of a family holiday to become eyewitness footage of a terrible tragedy.

Today, it’s a completely different situation. During almost every news event, bystanders use their mobile phones to share text updates in real-time on social media, as well as to capture and post pictures and videos straight to Twitter, Facebook, Instagram or YouTube.

But just because we now take this behavior for granted doesn’t mean we’ve worked out the rules for how to use this material legally, ethically or even logistically. Organizations are still working through the most appropriate ways to use this type of content. This is true whether it’s news outlets, brands, human rights groups or educators.

There are important differences between footage that has been sent directly to a particular organization versus material that has been uploaded publicly on a social network. The most important point to remember is that when someone uploads a photograph or video to a social network, the copyright remains with them. So if you want to download the picture or video to use elsewhere, you must first seek permission. If you simply want to embed the material, using the embed code provided by all of the social networks, legally you don’t need to seek permission. Ethically, however, it might be appropriate to contact the person who created the content to let them know how and where you intend to use it.

Seeking permission

A lawyer would always prefer an agreement to be conducted formally via a signed contract; however, in the heat of a breaking news event, seeking permission on the social network itself has become the norm. This has many benefits, not the least of which is that it provides an opportunity for immediate dialogue with the user who has shared the material.

Asking the right questions at the point of contact will help with your verification processes. The most important question to ask is whether the person actually captured the material him or herself. It is amazing how many people upload other people’s content on their own channels. They will often “give permission” for use even though they have no right to do so. You also want to ask basic questions about their location, and what else they could see, to help you authenticate what they claim to have witnessed.

If the person has just experienced a traumatic or shocking event, they could possibly still be in a dangerous situation. Establishing that they are safe and able to respond is also a crucial step. When seeking permission, it’s also important to be as transparent as possible about how you intend to use the footage. If you intend to license the video globally, this should be explained in a way that ensures that the uploader understands what that means.

Here’s one example of how to do it:

However, if you want a watertight legal agreement, you would need to arrange something more substantial over email. If you do seek permission on the social network itself, make sure that you take a screenshot of the exchange. People will sometimes provide permission for use, and then, after negotiating an exclusive deal with another organization, they will delete any exchanges on social media that show them giving permission to others.

Payment

There isn’t an industry standard for payment. Some people want payment for their material, and others don’t. Some people are happy for organizations to use their photo or video, as long as they are credited. Other people don’t want to be credited.

This is why you should ask these questions when you are seeking permission. You should also think about the implications of using the material. For example, a person might have captured a piece of content and in their mind, they’ve only shared it with their smallish network of friends and family. But they didn’t expect a journalist to find it. They captured it when they were perhaps somewhere they shouldn’t have been, or they captured something illegal and they don’t want to be involved. Or they simply don’t want a picture, quickly uploaded for their friends to see, to end up embedded on an online news site with millions of readers.

Here’s an example of a response from a person who uploaded a picture to Instagram during the shooting at the Canadian parliament in October 2014.

As part of ongoing monitoring and research, Eyewitness Media Hub has analyzed hundreds of exchanges between journalists and uploaders over 18 months in 2013, 2014 and 2015, and the responses of the people who created the material are not always what you would expect. This piece by Eyewitness Media Hub, of which I’m a co-founder, reflects on the content that emerged during the Paris shootings in early 2015, and the people who found themselves and their material unexpectedly at the center of the news coverage.

Crediting

Our experience and analysis show that the vast majority of people don’t want payment; they simply want a credit. This isn’t just a case of what’s right: It’s also a question of being transparent with the audience. There isn’t an industry standard when it comes to crediting, as every uploader wants to be credited in a different way. Especially if you’re not paying to use their material, you have a legal right to follow their instructions.

With television news, without the opportunity to embed content, a credit should be added onscreen. The most appropriate form of credit is to include two pieces of information. First, the social network where the footage was originally shared, and, second, the person’s name, in the way they asked to be credited. That might be their real name or their username, e.g., Twitter / C. Wardle or Instagram / cward1e or YouTube / Claire Wardle.

Online, the content should be embedded from the platform that it was originally posted, whether that’s Twitter, Instagram or YouTube. That means the credit is there as part of the embed. If a screen grab is taken of a picture or a video sourced from a social network, the same approach should be used. In the caption, it would be appropriate to hyperlink to the original post.

Be aware that embedded content will disappear from your site if it is removed from the social network by the original uploader. So you should ultimately try to procure the original file, especially if you are planning to run the content for a long time.

In certain situations, it’s necessary to use your judgment. If a situation is ongoing, then sharing the information of the person who created the content might not be the most sensible thing to do, as shown by this BBC News journalist:

Labeling

It is best practice to “label” who has captured the content. If we take this picture of a woman in the snowstorm in the Bekaa Valley in Lebanon, it’s important that the audience knows who took this. Was it a UNHCR staff member? Was it a freelance journalist? Was it a citizen journalist? Was it a refugee?

In this case a refugee took the photograph, but it was distributed by UNHCR to news organizations via a Flickr account. When someone unrelated to the newsroom takes a picture that is used by the newsroom, for reasons of transparency, any affiliation should be explained to the audience. Simply labeling this type of material as “Amateur Footage” or something similar doesn’t provide the necessary context.

Verification

There is no industry standard when it comes to labeling something as verified or not. The AP will not distribute a photograph or video unless it passes its verification procedures. While other news outlets try not to run unverified footage, it is difficult to be 100 percent sure about a photo or video that has been captured by someone unrelated to the newsroom.

As a result, many news organizations will run pictures or videos with the caveat that “this cannot be independently verified.” This is problematic, as the truth is that the newsroom may have run many verification checks, or relied on agencies to do these checks, before broadcasting or publishing a photo or video. So this phrase is being used as an insurance policy.

While research needs to explore the impact of this phrase on the audience, repeating it undermines the verification processes that are being carried out. Best practice is to label any content with the information you can confirm, whether that’s source, date or location. If you can confirm only two out of the three, add this information over the photo or video. We live in an age where audiences can often access the same material as the journalist; the audience is being exposed to the same breaking news photos and images in their social feeds. So the most important role for journalists is to provide the necessary context about the content that is being shared: debunk what is false, and provide crucial information about time, date or location, as well as showing how this content relates to other material that is circulating.

Being ethical

Overall, remember that when you work with material captured by others, you have to treat the content owner with respect, you need to work hard to verify what is being claimed, and you need to be as transparent as possible with your audience.

The people uploading this phone-taken footage are mostly eyewitnesses to a news event. They are not freelancers. The majority wouldn’t identify themselves as citizen journalists. They often have little knowledge of how the news industry works. They don’t understand words like exclusively, syndication or distribution.

Journalists have a responsibility to use the content ethically. Just because someone posted a piece of content publicly on a social network does not mean that they have considered the implications of its appearing on a national or international news outlet.

You must seek informed consent, not just consent, meaning: Does the uploader understand what they’re giving permission for? And when it comes to crediting, you must talk to them about whether and how they would like credit. The responses are constantly surprising.

Written by Mohammed Haddad

When you look at the chart below (Figure 4.1), you will see a series of steady orange and black bars followed by a large spike in 2016.

Once you take a closer look at the caption you will understand that this chart shows the number of structures destroyed and people affected by Israel’s policy of home demolitions.

As Nathan Yau, author of Flowing Data, put it, “data is an abstraction of real life” (2013). Each number represents a family, and each number tells a story.

“Broken Homes” is the most comprehensive project to date tracking home demolitions in East Jerusalem, a Palestinian neighbourhood that has been occupied by Israel for 50 years.1

Working closely with the United Nations, Al Jazeera tracked every single home demolition in East Jerusalem in 2016. It turned out to be a record year, with 190 structures destroyed and more than 1,200 Palestinians displaced or affected.

We decided to tackle this project after witnessing an escalation in violence between Israelis and Palestinians in late 2015.

The goal was twofold: To understand how Israel's home demolitions policy would be affected by the increased tensions, and to tell readers the human stories behind the data.

The project reveals the impact on Palestinian families through video testimony, 360-degree photos and an interactive map that highlights the location, frequency and impact of each demolition.

Our producer in Doha began coordinating with the UN in late 2015 to develop a framework for the project. The UN routinely gathers data on home demolitions, and while some of it is available online, other aspects—including GPS coordinates—are only recorded internally.

We wanted to be able to show every demolition site on a map, so we began obtaining monthly data sets from the UN. For each incident, we included the demolition date, number of people and structures affected, a brief description of what happened, and a point on our East Jerusalem map showing the location.

We cross-checked these with news reports and other local information about home demolitions. We then selected a case to highlight each month, as a way of showing different facets of the Israeli policy—from punitive to administrative demolitions, affecting everyone from young children to elderly residents.

Our reporter on the ground travelled throughout East Jerusalem over the course of the year to speak with many of the affected families, in order to explore their losses in greater depth and to photograph and record the demolition sites.

There was a broad range of responses from the affected families. The interviews had to take place in the physical location of the demolition, which could be a difficult experience for those affected, so sensitivity and patience were required at all stages, from setting up the meetings to recording the material.

On the whole, the families responded well to the project. They were very generous with their time and in sharing their experiences.

In one instance, a man had written down a list of things he wanted to say to us. In another case, it took a few attempts to convince the family to take part. One family declined to meet with us and so we had to liaise with the UN and f ind another family willing to speak about their home demolition.

Many news organizations, including Al Jazeera, have reported on individual home demolitions over the years. One of the main reasons for taking a data-driven approach this time was to clearly contextualize the scale of the story by counting each and every demolition. This context and fresh perspective are especially important when reporting on an ongoing topic to keep readers engaged.

A word of advice for aspiring data journalists: Taking a data-driven approach to a story doesn’t need to be technical or expensive.

Sometimes simply following and counting occurrences of an event over time tells you a lot about the scale of a problem. As long as your data-gathering methodology remains consistent, there are many stories that you can tell using data that you might not otherwise report on. Also, be patient. We gathered data for an entire year to tell this story.

The most important thing is to thoroughly storyboard exactly what data you need before sending any reporters out into the field. Most of the time you won’t need any special equipment either.

We used an iPhone to take all the 360-degree images and capture the specific GPS coordinates.

The project—released in January 2017 in English, Arabic and Bosnian— presents a grim warning about what lies ahead as Israel continues to deny building permits to 98% of Palestinian applicants, ramping up the pressure on a large and growing population.

Footnotes

1. interactive.aljazeera.com/aje/2017/jerusalem-2016-home-demolitions/index.html

Works Cited

Yau, N. (2013, June 28). Understanding data—Context. Big Think. bigthink.com/experts-corner/understanding-data-context

We asked some of data journalism’s leading practitioners and proponents why they think data journalism is an important development. Here is what they said.

Filtering the Flow of Data

When information was scarce, most of our efforts were devoted to hunting and gathering. Now that information is abundant, processing is more important. We process at two levels: (1) analysis to bring sense and structure out of the never-ending flow of data and (2) presentation to get what’s important and relevant into the consumer’s head. Like science, data journalism discloses its methods and presents its findings in a way that can be verified by replication.

New Approaches to Storytelling

Data journalism is an umbrella term that, to my mind, encompasses an ever-growing set of tools, techniques and approaches to storytelling. It can include everything from traditional computer-assisted reporting (using data as a ‘source’) to the most cutting edge data visualization and news applications. The unifying goal is a journalistic one: providing information and analysis to help inform us all about important issues of the day.

Like Photo Journalism with a Laptop

‘Data journalism’ only differs from ‘words journalism’ in that we use a different kit. We all sniff out, report, and relate stories for a living. It’s like ‘photo journalism’; just swap the camera for a laptop.

Data Journalism is the Future

Data-driven journalism is the future. Journalists need to be data-savvy. It used to be that you would get stories by chatting to people in bars, and it still might be that you’ll do it that way some times. But now it’s also going to be about poring over data and equipping yourself with the tools to analyze it and picking out what’s interesting. And keeping it in perspective, helping people out by really seeing where it all fits together, and what’s going on in the country.

Number-Crunching Meets Word-Smithing

Data journalism is bridging the gap between stat technicians and wordsmiths. Locating outliers and identifying trends that are not just statistically significant, but relevant to de-compiling the inherently complex world of today.

Updating Your Skills Set

Data journalism is a new set of skills for searching, understanding and visualizing digital sources in a time that basic skills from traditional journalism just aren’t enough. It’s not a replacement of traditional journalism, but an addition to it.

In a time where sources go digital, journalists can and have to be closer to those sources. The Internet opened up possibilities beyond our current understanding. Data journalism is just the beginning of evolving our past practices to adapt to the online.

Data journalism serves two important purposes for news organizations: finding unique stories (not from news wires) and execute your watchdog function. Especially in times of financial peril, these are important goals for newspapers to achieve.

From the standpoint of a regional newspaper, data journalism is crucial. We have the saying ‘a loose tile in front of your door is considered more important than a riot in a far-away country’. It’s hits you in the face and impacts your life more directly. At the same time, digitisation is everywhere. Because local newspapers have this direct impact in their neighbourhood and sources become digitalised, a journalist must know how to find, analyze and visualise a story from data.

A Remedy for Information Asymmetry

Information asymmetry — not the lack of information, but the inability to take in and process it with the speed and volume that it comes to us — is one of the most significant problems that citizens face in making choices about how to live their lives. Information taken in from print, visual and audio media influence citizens' choices and actions. Good data journalism helps to combat information asymmetry.

An Answer to Data-driven PR

The availability of measurement tools and their decreasing prices, in a self-sustaining combination with a focus on performance and efficiency in all aspects of society, have led decision-makers to quantify the progresses of their policies, monitor trends and identify opportunities.

Companies keep coming up with new metrics showing how well they perform. Politicians love to brag about reductions in unemployment numbers and increases in GDP. The lack of journalistic insight in the Enron, Worldcom, Madoff or Solyndra affairs is proof of many a journalist’s inability to clearly see through numbers. Figures are more likely to be taken at face value than other facts as they carry an aura of seriousness, even when they are entirely fabricated.

Fluency with data will help journalists sharpen their critical sense when faced with numbers and will hopefully help them gain back some terrain in their exchanges with PR departments.

Providing Independent Interpretations of Official Information

After the devastating earthquake and subsequent Fukushima nuclear plants disaster in 2011, the importance of data journalism has been driven home to media people in Japan, a country which is generally lagging behind in digital journalism.

We were at a loss when the government and experts had no credible data about the damage. When officials hid SPEEDI data (predicted diffusion of radioactive materials) from the public, we were not prepared to decode it even if it were leaked. Volunteers began to collect radioactive data by using their own devices but we were not armed with the knowledge of statistics, interpolation, visualization and so on. Journalists need to have access to raw data, and to learn not to rely on official interpretations of it.

Dealing with the Data Deluge

The challenges and opportunities presented by the digital revolution continue to disrupt journalism. In an age of information abundance, journalists and citizens alike all need better tools, whether we’re curating the samizdat of the 21st century in the Middle East, processing a late night data dump, or looking for the best way to visualise water quality for a nation of consumers. As we grapple with the consumption challenges presented by this deluge of data, new publishing platforms are also empowering everyone to gather and share data digitally, turning it into information. While reporters and editors have been the traditional vectors for information gathering and dissemination, the flattened information environment of 2012 now has news breaking first online, not on the news desk.

Around the globe, in fact, the bond between data and journalism is growing stronger. In an age of big data, the growing importance of data journalism lies in the ability of its practitioners to provide context, clarity and, perhaps most important, find truth in the expanding amount of digital content in the world. That doesn’t mean that the integrated media organizations of today don’t play a crucial role. Far from it. In the information age, journalists are needed more than ever to curate, verify, analyze and synthesise the wash of data. In that context, data journalism has profound importance for society.

Today, making sense of big data, particularly unstructured data, will be a central goal for data scientists around the world, whether they work in newsrooms, Wall Street or Silicon Valley. Notably, that goal will be substantially enabled by a growing set of common tools, whether they’re employed by government technologists opening Chicago, healthcare technologists or newsroom developers.

Our Lives are Data

Good data journalism is hard, because good journalism is hard. It means figuring out how to get the data, how to understand it, and how to find the story. Sometimes there are dead ends, and sometimes there’s no great story. After all if it were just a matter of pressing the right button, it wouldn’t be journalism. But that’s what makes it worthwhile, and — in a world where our lives are increasingly data — essential for a free and fair society.

A Way to Save Time

Journalists don’t have time to waste transcribing things by hand and messing around trying to get data out of PDFs, so learning a little bit of code, or knowing where to look for people who can help, is incredibly valuable.

One reporter from Folha de São Paulo was working with the local budget and called me to thank us for putting up the accounts of the municipality of São Paolo online (two days work from a single hacker!). He said he had been transcribing them by hand for the past three months, trying to build up a story. I also remember solving a ‘PDF issue’ for ‘Contas Abertas’, a parliamentary monitoring news organisation: 15 minutes and 15 lines of code solved a months worth of work.

An Essential Part of the Journalists' Toolkit

I think it’s important to stress the “journalism” or reporting aspect of ‘data journalism’. The exercise should not be about just analyzing data or visualizing data for the sake of it, but to use it as a tool to get closer to the truth of what is going on in the world. I see the ability to be able to analyze and interpret data as an essential part of today’s journalists' toolkit, rather than a separate discipline. Ultimately, it is all about good reporting, and telling stories in the most appropriate way.

Data journalism is another way to scrutinise the world and hold the powers that be to account. With an increasing amount of data available, now more than ever it is important that journalists are of aware of data journalism techniques. This should be a tool in the toolkit of any journalist: whether learning how to work with data directly, or collaborating with someone who can.

Its real power is in helping you to obtain information that would otherwise be very difficult to find or to prove. A good example of this is Steve Doig’s story that analyzed damage patterns from Hurricane Andrew. He joined two different datasets: one mapping the level of destruction caused by the hurricane and one showing wind speeds. This allowed him to pinpoint areas where weakened building codes and poor construction practices contributed to the impact of the disaster. He won a Pulitzer Prize for the story in 1993 and it’s great inspiration of what is possible.

Ideally you use the data to pinpoint outliers, areas of interest, or things which are surprising. In this sense data can act as a lead or a tip off. While numbers can be interesting, just writing about the data is not enough. You still need to do the reporting to explain what it means.

Adapting to Changes in Our Information Environment

New digital technologies bring new ways of producing and disseminating knowledge in society. Data journalism can be understood as the media’s attempt to adapt and respond to the changes in our information environment — including more interactive, multi-dimensional story-telling, enabling readers to explore the sources underlying the news and encouraging them to participate in the process of creating and evaluating stories.

A Way to See Things You Might Not Otherwise See

Some stories can only be understood and explained through analyzing — and sometimes visualizing — the data. Connections between powerful people or entities would go unrevealed, deaths caused by drug policies that would remain hidden, environmental policies that hurt our landscape would continue unabated. But each of the above was changed because of data that journalists have obtained, analyzed and provided to readers. The data can be as simple as a basic spreadsheet or a log of cell phone calls, or complex as school test scores or hospital infection data, but inside it all are stories worth telling.

A Way To Tell Richer Stories

We can paint pictures of our entire lives with our digital trails. From what we consume and browse, to where and when we travel, to our musical preferences, our first loves, our children’s milestones, even our last wishes – it all can be tracked, digitised, stored in the cloud and disseminated. This universe of data can be surfaced to tell stories, answer questions and impart an understanding of life in ways that is currently surpassing even the most rigorous and careful reconstruction of anecdotes.

Written by Aika Rey

Abstract

How a data story on road crash incidents in the Cagayan province in the Philippines led to positive policy and social change.

Keywords: data journalism, road safety, community engagement, mapping, Philippines, policy and social change

Data shows that fatalities from vehicular crash incidents in the Philippines have been increasing steadily over the years. Injuries from road crash incidents are now a top cause of death among Filipino youth.

Because of this, we built a microsite that compiled relevant information on road safety. We gathered and analyzed data, derived insights, published stories, and designed civic engagement opportunities—both on the ground and on digital—in order to educate the public about road safety.1

We also started running a video series entitled “Right of Way” which tackles motorist and commuter issues in Metro Manila. That is how Rappler’s #SaferRoadsPH campaign was born.

Compiling relevant data about road traffic deaths and injuries was a challenge. With no comprehensive national database on road crash numbers, we knocked on doors and gathered data from over a dozen national and local government units, including police offices in various cities and provinces.

Data acquired from these repositories are not standardized. A significant part of the work involved cleaning the data for analysis. One big challenge was how to map data when location information is either incomplete or not consistently recorded.2

Using the open-source data-cleaning application OpenRefine, we were able to come up with a normalized database of information acquired from the different government agencies. This allowed us to determine locations, dates and the number of people affected by crash incidents. Although still incomplete, our collection is probably the biggest single compilation of data on road crash incidents in the Philippines at the moment.

But what made our approach distinctive is that on top of stories, analysis and visualizations based on our collection of data, we made the effort to present them directly to communities concerned not just online but in on-the-ground activities. In the process, data analytics led to civic engagement activities.

One particular story that stood out in our coverage was the in-depth story on the Cagayan province, located roughly 600 km north of Manila, which is the area most affected by vehicular crash fatalities. We visited key offices in the province to get road crash incident data, as well as to conduct interviews with victims, local police and public service officials.

Following this exercise, in June 2017, Rappler conducted a road safety awareness forum in the province’s capital city Tuguegarao to present our findings. The forum sought to educate the public about road safety concerns in the province, as well as influence key officials to address the policy gap.3

Apart from graphs showing the times of day when critical incidents occur the most, at the forum we presented a heat map, created using Google Fusion Tables, which shows the locations with the greatest number of incidents in the Cagayan province (Figure 5.1).

Officials present attributed these numbers, among others, to the absence of pedestrian lanes. A check of schools in the city showed no pedestrian lanes in front of schools. After the forum, a social experiment was conducted where locals sketched pedestrian lanes using chalk in front of a school. Law enforcement officials wanted to see if motorists would stop at pedestrian lanes as students cross. Rappler later posted a video story on Facebook about this experiment.4

The video generated a lot of interest. A Rappler reader who saw the video reached out to us and volunteered to provide paint for pedestrian lanes within the city. Months later, through the combined efforts of local government and volunteers, schools within the city finally got pedestrian lanes. The painting project was completed on 30 September 2017. Two years later, the city government approved a local ordinance on road safety.

This project showed that data-driven reporting need not end when the editor clicks publish. It is proof that combining data journalism with online and offline community engagement can lead to positive policy and social change.

Footnotes

1. www.rappler.com/saferroadsph

2. We published a full explanation of our data sources here: r3.rappler.com/move-ph/issues/road-safety/171657-road-crash-numbers-data-sources

3. r3.rappler.com/move-ph/issues/road-safety/171778-road-crash-incidents-cagayan-valley

4. r3.rappler.com/move-ph/issues/road-safety/172432-cagayan-police-pedestrian-lane-chalk

Written by Dr. Hauke Janssen

It began with a cardboard box full of newspaper clippings. In 1947, Rudolf Augstein, the founder and publisher of Der Spiegel, mandated that his publication should gather and maintain an archive of previously published work.

That box soon grew to become an archive spanning hundreds, then thousands of meters of shelves. Newspapers, magazines and other news media were catalogued, along with original documents from government departments and other sources. Augstein praised his archive, which he said “can conjure up the most extravagant information.” He died in 2002.

More than any other republisher in Germany, Augstein believed in the power and value of maintaining an archive, and in the importance of applying it to a fact-checking process.

Up to the late 1980s, Spiegel’s archive was purely paper based. Beginning in the 1990s, the classic archives expanded into the virtual space. Today, the archive adds 60,000 new articles each week in its custom Digital Archive System (Digas). This information is collected from over 300 sources reviewed on a regular basis, which includes the entire national German press as well as several international publications. Digas currently stores more than 100 million text files and 10 million illustrations.

From an archive to a documentation department

A mistake led Der Spiegel to the realization that fact-checking is necessary. When an archivist pointed out a serious error in an article that had already been printed, Augstein answered gruffly, “Well, check that in the future earlier, then.“

From that point forward, fact-checking became a part of the duties of archive employees. In June 1949, Spiegel issued guidelines to all its journalists that outlined the necessity that every fact be checked. The guidelines read in part:

"Spiegel must contain more personal, more intimate and more background information than the daily press does ... All news, information and facts that Spiegel uses and publishes must be correct without fail. Each piece of news and each fact must be checked thoroughly before it is passed on to the news staff. All sources must be identified. When in doubt, it is better not to use a piece of information rather than to run the risk of an incorrect report."

Hans D. Becker, the magazine’s managing editor in the 1950s, described the change from a traditional archive to a documentation department.

“Originally, the news library was only supposed to collect information (mostly in the form of press clippings),” he said. “What started as collecting on the dragnet principle imperceptibly became information-gathering through research. Amidst the ‘chaos of the battlefield’ of a newsroom, collecting and researching information for use in reporting imperceptibly became the exploitation of what was collected and gathered to prove what was claimed ...”

How Spiegel does fact-checking today

The Dok, as we call it, is today organized into sections, called “referats,” that correspond to the various desks in the news departments, such as politics, economy, culture, science, etc. It employs roughly 70 “documentation journalists.” These are specialists who often possess a doctorate in their respective fields, and include biologists, physicists, lawyers, economists, MBAs, historians, scholars of Islam, military experts and more.

They are charged with checking facts and with supporting our journalists by providing relevant research. As soon as the author’s manuscript is edited, the page proof is transferred to the relevant Dok-Referat. Then the fact-checking starts.

Spiegel has very specific and detailed guidelines for fact-checking. This process ensures we apply the same standard to all work, and helps ensure we do not overlook key facts or aspects of a story. Dok- Referats use the same markings on manuscripts, creating a level of consistency that ensures adherence to our standards.

This approach can be applied to any story, and is particularly useful in investigative work, which must meet the highest standards.

Some of the key elements of our guidelines:

• Any fact that is to be published will be checked to see if it is correct on its own and in context, employing the resources at hand and dependent on the time available.

• Every verifiable piece of information will be underlined.
  

• Standardized marks will be used to denote statements as correct, incorrect, not verifiable, etc.

• Correct facts and figures will be checked off. If corrections are necessary, they will be noted in red ink in the margin, using standard proofreading marks.

• The source of factual corrections and quotations must be given.

• Corrections accepted by the author(s) will be checked off, the others will be marked n.ü. (not accepted).

• When fact-checking a manuscript, other and if possible more accurate sources than the author‘s sources should be used.

• A statement is considered verified only if confirmed by reliable sources or experts.

• If a piece of research contradicts an author’s statement, the author must be notified of the contradiction during the discussion of the manuscript. If a fact is unverifiable, the author must also be notified.

• A journalist’s source who is the object of an article may be contacted only with permission from the author. (In practice, we often speak with sources to check facts.)

• Complex passages will be double-checked by the documentation department specialized in the subject matter.

• Sometimes the limited time available means that priorities must be set. In such cases, facts that are the clear responsibility of the fact-checker must be checked first, particularly:

-Are the times and dates correct?

-Does the text contradict itself?

-Are the names and offices/jobs correct?

-Are the quotations correct (in wording and in context)? How current and trustworthy are the sources used?

-How current and trustworthy are the sources used?

The above list represents the most critical elements to be verified in an article when there is limited time for fact-checking. Newsrooms that do not have a similar documentation department should emphasize that reporters and editors double-check all of these items in any story prior to publication.

Evaluating Sources

Fact-checking starts with comparing a story draft with the research materials provided by the author. The fact-checker then seeks to verify the facts and assertions by gathering additional sources that are independent of each other. For crucial passages, the checker examines a wide variety of sources in order to examine what is commonly accepted and believed and what is a more subjective or biased point of view. They determine what is a matter of fact and what is controversial or, in some cases, a myth.

We use our Digas database to surface relevant and authoritative sources. It’s also the responsibility of every Spiegel fact-checker to study the relevant papers, journals, studies, blogs, etc. in their field, daily. This ensures that they have current knowledge on relevant topics, and that they know the trustworthiness of different sources.

This form of domain expertise is essential when evaluating the credibility of sources. However, there are some general guidelines that can be followed when evaluating sources:

• Prefer original documents. If an academic study is quoted, obtain the original, full text. If company earnings are cited, obtain their financials. Do not reply on press summaries and press releases when the original document can be obtained.

• Prefer sources that delineate between facts and opinion, and that supply facts in their work.

• Prefer sources that clearly indicate the source of their information, as this enables you to verify their work. (Media outlets or other entities that overly rely on anonymous sources should be treated with caution.)

• Beware of sources that make factual errors about basic facts, or that confuse basic concepts about a subject matter.

Examples of checked manuscripts

After an article has been checked at Spiegel, the documentarist and the author discuss possible corrections until they agree on the final version. The author makes the corrections to the manuscript. The fact-checker checks the corrections a second time and also any other changes that may have been made in the meantime.

Accuracy is the basic prerequisite for good journalism and objective reportage. Journalists make mistakes, intended or not. Mistakes damage the most valuable asset of journalism: credibility. That is, after all, the quality to which journalists refer most frequently to distinguish their journalism.

One method to reduce the probability of mistakes is verification; that is, checking facts before publication.

A 2008 thesis produced at the University of Hamburg counted all the corrections made by the documentation department in a single issue of Der Spiegel. The final count was 1,153. Even if we exclude corrections related to spelling and style, there were still 449 mistakes and 400 imprecise passages, of which more than 85 percent were considered to be relevant or very relevant.

Abstract

Documenting worker deaths in Turkey to advocate for improved working conditions.

Keywords: Turkey, Soma, labour conditions, data journalism, open data, freedom of information (FOI)

In the wake of the Soma mine disaster in Turkey in 2014, it transpired that it was extremely difficult to document the conditions of workers.

There were discrepancies with figures on worker unionization and a scarcity of data on worker deaths over previous decades. What was available was often disorganized and lacking in detail. We wanted to make this data widely accessible and shed light on the deaths of workers in other sectors.

With this in mind, a programmer, an editor and myself developed the “Open Database of Deceased Workers in Turkey,” a project hosted by the data journalism portal Dağ Medya, that gathered data from multiple sources, verified it, and made it available for everyone to access and use.1

In Turkey at least 130 workers die per month from a variety of causes. The most important goal of the project was to raise awareness of these deaths and their frequency, as well as to publicly recognize victims and the poor working conditions that they endured.

The project comprised embeddable maps, graphs and data in different formats.2

It covered the deaths of workers in over 20 sectors from 2011 to 2014. After the project was completed, we continued to report the death of workers through regular media monitoring each month. Crucially, the database includes the names of the companies that employed them (Figure 6.1).

The project began in 2015. We started by submitting freedom of information (FOI) requests and collecting data from trusted NGOs that were extracting data from various sources and were making it publicly accessible.

The first challenge we encountered was that it was not easy to get open data through FOI requests. Sometimes it took two weeks, sometimes four months, to obtain data through our requests.

Next, a more unexpected challenge arose, which I am recounting because it surfaces conflicting perspectives on this type of work. When we announced the project, one of the projects whose data we were using—İSİG Meclisi (Health and Safety Labour Watch)—became unhappy about us using it.3

The reason was, they claimed, that our project simply republished data that they had gathered. They understood our using of their data in this way as taking advantage of their labour.

The opposition to the project persisted, in spite of our asking for permission and the fact that their data is publicly available. While they accused us of “pornifying” workers’ deaths with our visualizations and tables, we saw our project as creating public value by increasing the outreach of the data they had collected, through visually accessible and readily downloadable formats.

While human stories are vital, we believed that unstructured, raw data was important in order to provide a more systematic view of these injustices. We found it difficult to reach consensus around this logic and struggled to make the case for the value of collaborative open data-sharing practices. We ended up publishing monthly worker deaths by comparing official data gathered through FOI requests with the data that we collected through our own monitoring of these cases.

Following this project, the institutions that we petitioned with FOI requests began to share their data in a more structured way and visualize it.

We took this as evidence that we had accomplished one of our goals: To make this information more widely accessible.

Ultimately, the project was recognized as a finalist in the 2015 Data Journalism Awards.

Footnotes

1.http://community.globaleditorsnetwork.org/content/open-database-deceased-workers-turkey-0

2. platform24.org/veri-gazeteciligi/451/turkiyede-isci-olumleri-veritabani-hazirlandi (Turkish language)

3. www.isigmeclisi.org(Turkish language)

We asked some of our contributors for their favorite examples of data journalism and what they liked about them. Here they are.

Do No Harm in the Las Vegas Sun

My favourite example is the Las Vegas Sun’s 2010 Do No Harm series on hospital care (see Figure 5). The Sun analyzed more than 2.9 million hospital billing records, which revealed more than 3600 preventable injuries, infections and surgical mistakes. They obtained data through a public records request and identified more than 300 cases in which patients died because of mistakes that could have been prevented. It contains different elements, including: an interactive graphic which allows the reader to see by hospital, where surgical injuries happened more often than would be expected; a map with a timeline that shows infections spreading hospital by hospital; and an interactive graphic that allows users to sort data by preventable injuries or by hospital to see where people are getting hurt. I like it because it is very easy to understand and navigate. Users can explore the data in a very intuitive way. Also it had a real impact: the Nevada legislature responded with six pieces of legislation. The journalists involved worked very hard to acquire and clean up the data. One of the journalists, Alex Richards, sent data back to hospitals and to the state at least a dozen times to get mistakes corrected.

Government Employee Salary Database

I love the work that small independent organizations are performing every day, such as ProPublica or the Texas Tribune who have a great data reporter in Ryan Murphy. If I had to choose, I’d pick the Government Employee Salary Database project from the Texas Tribune (Figure 6). This project collects 660,000 government employee salaries into a database for users to search and help generate stories from. You can search by agency, name or salary. It’s simple, meaningful and is making inaccessible information public. It is easy to use and automatically generates stories. It is a great example of why the Texas Tribune gets most of its traffic from the data pages.

Full-text visualization of the Iraqi War Logs, Associated Press

Jonathan Stray and Julian Burgess’ work on Iraq War Logs is an inspiring foray into text analysis and visualization using experimental techniques to gain insight into themes worth exploring further within a large textual dataset (Figure 7).

By means of text-analytics techniques and algorithms, Jonathan and Julian created a method that showed clusters of keywords contained in thousands of US-government reports on the Iraq war leaked by Wikileaks in visual form.

Though there are limitations to the methods presented and the approach is experimental, it presents an innovative approach. Rather than trying to read all the files or reviewing the War Logs with a preconceived notion of what may be found by inputting particular keywords and reviewing the output, this technique calculates and visualises topics/keywords of particular relevance.

With increasing amounts of data — both textual (emails, reports, etc.) and numeric — coming into the public domain, finding ways to pinpoint key areas of interest will become more and more important — it is an exciting sub-field of data journalism.

Murder Mysteries

One of my favorite pieces of data journalism is the “Murder Mysteries” project by Tom Hargrove of the Scripps Howard News Service (Figure 8). He built from government data and public records requests a demographically-detailed database of more than 185,000 unsolved murders, and then designed an algorithm to search it for patterns suggesting the possible presence of serial killers. This project has it all: hard work gathering a database better than the government’s own, clever analysis using social science techniques, and interactive presentation of the data online so readers can explore it themselves.

Message Machine

I love ProPublica’s Message Machine story and nerd blog post (Figure 9). It all got started when some Twitterers expressed their curiosity about having received different emails from the Obama campaign. The folks at ProPublica noticed, and asked their audience to forward any emails they got from the campaign. The presentation is elegant, a visual diff of several different emails that were sent out that evening. It’s awesome because they gathered their own data (admittedly a small sample, but big enough to tell the story). But it’s even more awesome because they’re telling the story of an emerging phenomena, big data used in political campaigns to target messages to specific individuals. It is just a taste of things to come.

Chartball

One of my favourite data journalism projects is Andrew Garcia Phillips' work on Chartball (Figure 10). Andrew is a huge sports fan with a voracious appetite for data, a terrific eye for design and the capacity to write code. With Chartball he visualises not only the sweep of history, but details the success and failures of individual players and teams. He makes context, he makes an inviting graphic and his work is deep and fun and interesting – and I don’t even care much for sports!

Written by Victor Ribeiro

On Oct 15, 2013, a 37-year-old activist named Jair Seixas (aka Baiano) was arrested as a protest supporting striking teachers was winding down in Rio de Janeiro. Seixas had been marching peacefully with eight human rights lawyers when police officers approached and accused him of setting fire to a police vehicle and minibus.

As he was being taken away, police refused to tell the lawyers which precinct he was being taken to, or what evidence they had of his alleged crimes.

Seixas was held in prison for 60 days and released. He continues to fight the charges brought against him. When his lawyers began to plan their defense strategy, they looked for videos that might help prove Seixas’ innocence. Their search involved looking on social networks, asking those who were at the event, and obtaining footage from the prosecution and courts.

They found five pieces of footage they felt had evidentiary value to their case. Two videos were official court records of the police officers’ testimonies under oath; two were videos submitted by the prosecution that were confirmed to have been filmed by undercover police officers who had infiltrated protesters; and the final clip was filmed by a media activist who was covering the protest and was present at the time of Seixas’ arrest. This activist used a cellphone to livestream the event, which provided a huge amount of critical first-hand footage of the event.

By putting these videos together, the lawyers found critical evidence of Seixas’ innocence. The filmed testimonies of the officers were full of contradictions and helped prove that the officers didn’t actually see Seixas set fire to the bus, contrary to what they had claimed earlier. The prosecution’s videos captured audio of undercover officers inciting protesters to violence. This helped demonstrate that, in some instances, the violence the protesters were being accused of had originated with undercover officers.

The final clip, filmed by a media activist, was the smoking gun: In a frame-by-frame analysis of roughly three hours of an archived livestream of the protest (324,000 frames!) the defense team uncovered a single frame of video that showed that the police vehicle Seixas was being accused of having set ablaze was the exact same vehicle that drove him away after he was detained. This was proven by comparing the identifying characteristics of the vehicle in the video with the one that Seixas was transported in.

We at WITNESS helped the defense identify and prepare this evidence, both by assembling screenshots of these videos into a storyboard as well as by editing a 10-minute evidentiary submission of video that was delivered to the judge, along with the accompanying documentation.

Though the case is still continuing, the evidence is clear and undeniable. This is an inspiring example of how video from both official and citizen sources can serve justice and protect the innocent from false accusations.

In less than a decade, newsgathering has been transformed by two significant developments.

The first is mobile technology. In the summer of 2013 an important tipping point was reached. For the first time, more than half (55 percent) of all new mobile phone handsets sold were smartphones.

By definition a smartphone has a high-quality camera with video capability, and it allows the user to easily connect to the Web to disseminate the pictures. As a result, more and more people have the technology in their pockets to very quickly film events they see around them, and share them directly with people who might be interested, as well as more widely via social networks.

The second, connected development is the social Web. When the BBC’s User Generated Content Hub started its work in early 2005, they were reliant on people sending content to one central email address. At that point Facebook had just over 5 million users, rather than the more than one billion today. YouTube and Twitter hadn’t launched. Now, every minute of the day, 100 hours of content is uploaded to YouTube, 250,000 tweets are sent and 2.4 million pieces of content are shared on Facebook. Audience behavior has shifted substantially.

Rather than film something and, when prompted, send it to a news organization, people shoot what they see and upload it to Facebook, YouTube or Twitter. Research has shown very few audience members have enough understanding of the news process to think of their footage as valuable enough to send it, unprompted, to a news organization or other entity. Essentially, they’re uploading the content to share the experience with their friends and family.

Increasingly, at any news event around the world there are “accidental journalists”: people standing in the right place at the right time with a smartphone in their hands. As Anthony De Rosa, the former social media editor for Reuters and current editor-in-chief of Circa, writes: “The first thought of the shooter is usually not: ‘I need to share this with a major TV news network’ because they don’t care about traditional television news networks or more likely they’ve never heard of them. They have, however, heard of the Internet and that’s where they decide to share it with the world.”

Similarly, during breaking news events, the audience is often more likely to turn to social networks for information, meaning first responders and emergency organizations are using social networks themselves. Unfortunately, these news events invite false information to circulate, either deliberately or by accident. Therefore, journalists and humanitarian professionals should always start from a position that the content is incorrect. During emergencies, when information can literally affect lives, verification is a critical part of the newsgathering and information dissemination process.

The importance of verification

The ability for anyone to upload content, and to label or describe it as being from a certain event, leaves many journalists, and particularly editors, terrified about being hoaxed or running with false content.

Some people go out of their way to deliberately hoax news organizations and the public by creating fake websites, inventing Twitter accounts, Photoshopping images or editing videos. More often, the mistakes that happen aren’t deliberate. People, trying to be helpful, often find mislabeled content from previous news events and share it. Below is an example of a man apologizing after tweeting a photo emailed to him by his wife. She had told him it showed Typhoon Usagi as it headed toward Hong Kong; in fact it was an old image of an- other event.

People downloading content from YouTube and uploading it to their own accounts, claiming it as their own, cause other problems. This isn’t a hoax - it’s what is known as a “scrape” - but it means we have to work harder to find the original uploader of the content.

The difficulty of finding original footage was demonstrated when the U.S. Senate Intelligence Committee released a playlist of 13 videos that had originally appeared on YouTube, which they had used to look for evidence related to the 2013 chemical weapons attack on East Gouta in Syria. A number of these videos were taken from a well-known Syrian aggregator YouTube channel which regularly republishes videos from other people’s channels. This suggested the videos within the playlist were not the original videos and were in fact “scrapes.” Using a range of different verification techniques, Félim McMahon from Storyful was able to discover the original versions of these videos. He wrote up the process here. What this example shows is that these issues are no longer just a concern for the journalism community.

Verification checks

Verification is a key skill, made possible through free online tools and old-fashioned journalism techniques. No technology can automatically verify a piece of UGC with 100 percent certainty. However, the human eye or traditional investigations aren’t enough either. It’s the combination of the two.

When a journalist or humanitarian professional finds a piece of information or content via social media, or has it sent to her, there are four elements to check and confirm:

1. Provenance: Is this the original piece of content?
2. Source: Who uploaded the content?
3. Date: When was the content created?
4. Location: Where was the content created?

1. Provenance: Confirming the authenticity of the piece of content

If you find content on a social media profile, you have to run a number of checks on that profile to make sure it is real.

In the case of a tweet, be aware that the site lemmetweetthatforyou.com makes it shockingly easy to fake a tweet, which can be then shared as a picture.

Another way people spread fake information on Twitter is by presenting the fake information as a retweet. For example: “Really? RT@JoeBiden I’m announcing my retirement from politics.” That makes it appear as if you’re simply retweeting an original tweet.

Fakers also often add a Twitter blue verification check mark to the cover photo on a faked account to make it appear legitimate. To check whether an account is actually verified, hover over the blue tick, and you will see the text “verified account” pop up. If it’s not there, it is not a verified account.

Facebook introduced a similar verification program, using the same blue tick system, for celebrities, journalists and government officials. Verified ticks can appear on Facebook pages as well as personal profiles. (As with Twitter, Facebook manages the verification program, and decides which verification requests to accept.) On Facebook pages, such as Usain Bolt’s below, the tick appears underneath the cover photo, next to the person’s name.

On personal profiles, the tick appears on the cover photo. Here’s the profile of Liz Heron, editor of emerging media at The Wall Street Journal:

It’s worth noting that, as with Twitter, people have been known to Photoshop blue ticks onto cover photos. So, as with Twitter, if you hover your mouse over the blue tick, the phrase “verified profile” will appear.

But as with Twitter, remember the verification process is far from transparent, so with less famous people, it can be unclear whether an unverified account is a fake, or whether they’re just not famous enough to be verified!

But even with these official verification programs in place, there is no quick way of checking whether an account is real, other than painstaking checks on all of the details available on the profile. Items to review include linked websites, location, previous pictures and videos, previous status updates or tweets. Who are their friends or followers? Who are they following? Do they feature on anyone else’s lists?

If you’re looking at a piece of rich content, such as a photo or video, one of the first questions is whether this is the original piece of footage or picture. Using reverse image search tools such as TinEye or Google Images3 you can find out whether it has been posted online previ- ously. (For more detail on using these tools, see Chapter 4 of this book.)

While deliberate hoaxes are rare, they do happen. In recent years there have been relatively harmless hoax videos produced by PR companies looking for publicity, and by students completing an end-of-term assignment. There have also been deliberate attempts to create false content, particularly in Syria and Egypt, where discrediting the “enemy” can be achieved via reputable-looking content shared on social media channels.

Techniques include creating a false, but identical-looking website and claiming responsibility for a bomb attack, or staging a gruesome incident and blaming the other side. Manipulation is relatively easy to do today, and whether you’re Nancy Pelosi trying to create a photograph of all female Congresswomen even when some of them are late, or a Syrian activist group sharing video of a man appearing to be buried alive, any journalist or humanitarian professional has to start off by assuming a piece of UGC is false. (See Chapter 5 of this book for more detail about verifying video.)

2. Confirming the source

The ultimate goal when attempting to verify UGC is to identify the original uploader and get in touch with them.

In that conversation, the key questions involve discovering where someone was standing when they took the footage, what they could see, and the type of camera used to record the footage. (These questions provide the essential data to answer Steve Buttry’s essential “How do you know that?” test outlined in the previous chapter.)

If someone is attempting to pass along false information, either deliberately or not, asking direct questions will often result in the person’s admission that they did not actually film the footage themselves. Additionally, it is possible to cross-reference answers to some of these questions with available information by examining the EXIF data in a photo, or comparing video of a specific location to Google Street View, which we detail in subsequent chapters.

But first you have to find the person responsible for the content. Researching the history of an uploader can mimic the characteristics of an old-fashioned police investigation, and perhaps also make you feel more like a stalker rather than a journalist or researcher.

Some people list a great deal of information on their social profiles, and a real name (especially one that is not too common) can provide a wealth of information. As people live more of their lives on different social networks, they are often unaware how clues can be combined to build up a substantial dossier of information. A YouTube profile with little personal information listed but that includes a website URL can lead a journalist to a person’s address, email and personal telephone number, via the website who.is.

3. Confirming the date of the event

Verifying the date of a piece of video can be one of the most difficult elements of verification. Some activists are aware of this fact and will show a newspaper from that day, with the date clearly visible when they share their footage. This obviously isn’t foolproof, but if an uploader becomes known and trusted by organizations, be they news or humanitarian, this is a helpful additional piece of information.

Be aware that YouTube date stamps its video using Pacific Standard Time. This can sometimes mean that video appears to have been uploaded before an event took place.

Another way to help ascertain date is by using weather information. Wolfram Alpha is a computational knowledge engine that, among other things, allows you to check weather from a particular date. (Simply type in a phrase such as “What was the weather in Caracas on September 24, 2013” to get a result.) This can be combined with tweets and data from local weather forecasters, as well as other uploads from the same location on the same day, to cross-reference weather.

4. Confirming the location

Only a small percentage of content is automatically geolocated, but mapping platforms - Google Maps, Google Earth, Wikimapia - of the first checks that need to be performed for video and photos, and it is quite incredible what can be located.d Geolocation is always more difficult, however, when the imaging is out of date, for example in Syria, subject to damage from bombs or shelling, or on Long Island after Hurricane Sandy.

Activists who are aware of the challenges of verification often pan upward before or after filming some footage to identify a building that could be located on a map, whether that’s a tall tower, a minaret or cathedral, or signpost. This is partly a result of news organizations’ asking activist groups to do this, as well as activists themselves sharing advice about best practice when uploading UGC.

Verification as process

Unfortunately, people often see verification as a simple yes/no action: Something has been verified or not.

In practice, as described above and in subsequent chapters, verification is a process. It is relatively rare that all of these checks provide clear answers. It is therefore an editorial decision about whether to use a piece of content that originates from a witness.

Two recent academic studies performed content analysis of output on the BBC and Al Jazeera Arabic. They found that while these verification checks are undertaken by editorial staff, and considered absolutely necessary, the results of the checks are rarely shared with the audience.

As Juliette Harkin concluded in her 2012 study, “[n]either BBC Arabic nor Al Jazeera Arabic explicitly mentioned in any of the programs or video packages that were evaluated whether the sources were verified or were reliable. The common on air explanation of ‘this footage cannot be verified,’ was absent in all the content evaluated for this study.”f

There are recent moves to increase transparency with the audience about the verification checks made by journalists when a piece of UGC is used by a news organization. The AP and BBC are both working toward making their verification processes clearer; in August 2013, the BBC said that since a comprehensive study into the use of UGC during the Arab Spring, “the BBC has adopted new wording for all user-generated footage where independent verification has not been possible,” letting its audience know what it knows.

It is likely that within the next few years, a new grammar of verification will emerge, with the audience expecting to be told what is known and what isn’t known about a piece of UGC sourced from social media. With the audience able to see the same footage as the news organizations and others that gather material from the crowd, this level of transparency and accountability is required.

Brandy Zadrozny is an investigative reporter for NBC News, where she mostly covers misinformation, disinformation and extremism on the internet.

Nearly every story I report involves social media sleuthing. From profile backgrounding to breaking news to longer investigations, social media platforms offer some of the best ways to learn about a subject’s real life — their family, friends, jobs, personal politics and associations — as well as a window into secret thoughts and hidden online identities.

It’s an incredible time to be a journalist; people increasingly live their lives online and tools to find and search a subject’s social profiles are ubiquitous. At the same time, both normal folks and bad actors are getting smarter about hiding their tracks. Meanwhile, social media platforms like Facebook have reacted to negative press about privacy breaches and harmful ideologies spread on their platform by closing down the tools that journalists and researchers have become reliant on to uncover stories and identify people.

In the following chapter I’ll show some core approaches for investigating social accounts. The tools are the ones currently in my rotation, but before long they’ll be killed by Facebook or replaced by something better. The reporters who are best at this work have their own processes and gadgets to get there, but really, as in any brand of reporting, obsession and (virtual) shoe leather yield the best results. Be prepared to read thousands of tweets, click until the end of the Google results, and dive down a social media rabbit hole if you want to collect the tiny biographical clues that will help you answer the question, “Who is this?”

Usernames

A username is sometimes all we have, which is fine, because it’s almost always where we start. Such was the case of a then-New Hampshire Republican state representative who built one of Reddit’s most popular and odious men’s communities. The investigation behind the unmasking of the architect of Reddit’s The Red Pill, now a quarantined community, started with the username “pk_atheist.”

Some people hold on to usernames, using them with minimal variations, across various platforms and email providers. The more security-focused, like the New Hampshire state representative, create and ditch usernames with each new endeavor.

Whatever the case, there are a few sites that you should feed the username you’re searching into.

First, I plug the username into Google. People — especially younger ones who eschew the larger social platforms — tend to leave a trail even in more unexpected places, including comment sections, reviews and forums, that can lead you to information and other accounts.

Along with a Google search, use proprietary services. They cost money and depending on your newsroom’s budget, you may or may not have access. Most shops have Nexis, which is great for public records and court documents but sadly lacking in the email/username department. It’s also useful for researching people only in the United States. Pipl and Skopenow are among the best tools I’ve found for cross referencing “real world” information like phone numbers and property records with online records like emails and usernames, and both work globally. These paid search engines often provide phone and property records, but they can also identify Facebook and LinkedIn profiles that remain even after an account has been closed. They also connect accounts that people have largely forgotten about, such as old blogs and even Amazon wish lists — a gold mine for learning about what a person reads, buys and wants. You also get a lot of false positives with these, so I tend to start my investigation with their results and continue with other means of verification.

When I find a username or email I think might belong to my subject, I plug it into an online tool like namechk or namecheckr that looks for username availability across multiple platforms. These tools are designed to be an easy way for marketers to see if a given username they’re planning to register is available across platforms. But they’re also useful for checking whether a username you’re investigating also exists elsewhere. Obviously, just because a username has been registered on multiple platforms doesn’t mean these accounts all belong to the same person. But it’s a great starting point to looking across platforms.

For further username checking, there’s haveibeenpwned.com and Dehashed.com, which search data breaches for user information and can be a quick way to validate an email address and provide new leads.

Photos

A username isn’t always enough to go on, and nothing persuades like a picture. Profile photos are another way to verify the identity of a person across different accounts.

Google’s reverse image search is fine, but often other search engines — especially Russia’s Yandex — may deliver better results. I use the Reveye Chrome extension, which allows me to right click on an image and search for its match across multiple platforms including Google, Bing, Yandex and Tineye. The Search by Image extension also has a neat capture function that allows you to search from an image within an image.

There are problems with reverse image searching, of course. The search engines referenced above do a poor job finding images across Twitter and are all but useless for turning up results from sites like Instagram and Facebook.

What I’m most often looking at are different images of people. I can’t count how many times I’ve squinted at my monitor and asked my colleagues, “Is this the same person?”

I just don’t trust my eyes. Identifying characteristics across photos like moles or facial hair or features is helpful; lately, I also like to check it with a facial recognition tool like Face++, which allows you to upload two photos and then gives a probability that those belong to the same person. In these examples, the tool was able to positively identify me in photos 10 years apart. It also identified my colleague Ben across social media profile pics on Twitter and Facebook while correctly noting that he is not, in fact, Ben Stiller.

If you’re chasing trolls or scammers, you might find they’ve put more effort into obscuring their profile photo, or they may use fake photos. That’s when editing the photo and flipping it might help reverse engineer their process.

It’s not just profile photos that can be signposts, however. As people become more aware of and concerned with their own privacy and that of their family, they’re still inclined to share photos of things they’re proud of. I’ve identified people by connecting photos of things like cars, homes or pets. In this sense, photos become a means to connect accounts and the people behind them to one another, enabling you to build out the network around your target. This is a core practice when investigating social media accounts.

For example, we were looking to confirm the social accounts of a man who shot and killed nine people outside of a bar in Dayton, Ohio. His Twitter account offered clues to his political ideology but his handle, @iamthespookster, was unique and didn’t resemble his real name, which had been released by authorities. The fact that one of his victims was his sibling, a transgender man whose name was not in public records and hadn’t come out to the wider world yet, further complicated identifying the key figures. But throughout his and his family’s profiles were pictures of a dog, a pet that appeared as the banner image of his transgender brother’s unreported account.

The dog wasn’t the only helpful detail in the previous image. That image came from the Ohio shooter’s father, and helped us verify his personal accounts and those belonging to his family.

If you have an account on Facebook or Twitter, I can probably tell you the day you were born, even if you don’t share it on your profile or post about it yourself. Since a date of birth is often one of the first identifying pieces of police-provided information in breaking news situations, a reliable way to verify a social media account is by scrolling to the month and day in question on a suspected account and looking out for birthday wishes. Even if their own pages are empty, often moms and dads (like Connor Betts’ above) will post about their children’s birthdays.

The same is true for Twitter, because who doesn’t love a birthday?

But it’s even easier to find an identifying post on Twitter, because its advanced search tool is among the best offered by social platforms. Although I rarely announce my birthday, if ever, I was able to find a birthday tweet from a loving colleague who outed me.

Birthdays are just one example. Weddings, funerals, holidays, anniversaries, graduations — nearly every major life marker is celebrated on social media. These provide an opening for searching and investigating an account.

You can search for these keywords and by other filters with Facebook search tools. They don’t get as much mileage as they did before the platform’s pivot to privacy, but they exist. One of my favorites is whopostedwhat.com.

Relationships

You can judge a person by the company they keep on social media. We can tell a lot about a person’s life and leanings by examining the people with whom they interact online.

When I first joined Twitter, I made my husband and best friend sign up too, just so they could follow me. I think about that when I’m looking into accounts for work. The platforms don’t want you to be alone, either, so when you first open an account, an algorithm powers up. Influenced by the contacts list in your phone, your appearance in the contact lists of existing accounts, your location and other factors, a platform will suggest accounts to follow.

Because of that truth, it’s always illuminating to look at an account’s earliest followers and friends. TweetBeaver is a good tool for investigating the connections between large accounts and for downloading things like timelines and favorites of smaller accounts. For larger datasets, I rely on a developer with API access.

Let’s take The Columbia Bugle, a popular far-right anonymous Twitter account that boasts that it was retweeted twice by Donald Trump’s account.

The earliest follows of Max Delarge, an account claiming to be the editor of The Columbia Bugle, are San Diego-specific news sources and San Diego-specific sports accounts. Since many of Columbia Bugle’s tweets include videos from San Diego Trump rallies and events at the University of California, San Diego, we can be fairly confident that the person behind the account lives near San Diego.

With a new investigation, I like to start at the beginning of someone’s Twitter history and work forward in time. You can get there by hand, with an assist from an autoscroller chrome extension, or you can use Twitter’s advanced search to limit the time frame to the first few months of an account’s existence.

Curiously, the first six months of this account shows zero tweets.

This suggests that the person behind The Columbia Bugle might have deleted his earlier tweets. To find out why that might be, I can tweak my search. Instead of tweets from the account I’ll look for any tweets mentioning The Columbia Bugle.

These conversations confirm that ColumbiaBugle erased its first year of tweets, but doesn’t tell us why and the first accounts that the account interacted with don’t offer many clues.

To find recently deleted tweets, you can search Google’s cache; older deleted tweets can also sometimes be accessed in the Internet Archive’s Wayback Machine or another archive. The manual archive site archive.is turns up several deleted tweets from where ColumbiaBugle participated in an event where college students wrote pro-Trump messages on their campuses. To see all the tweets someone may have archived from that account, as I did to find this tweet, you can search by URL prefix, using an asterisk after the account name like this:

It’s rare for someone to successfully keep their real life separate from their online activities. For example, my NBC News colleague and I told the story of 2016’s most viral — and misleading — Election Day voter fraud claim, with an assist from a neighborhood acquaintance of the far-right troll who tweeted it.

Though the tweet originated with a man known to his followers as @lordaedonis, people from his actual neighborhood had responded to past tweets with his real name, which we included in a profile of an attention-hungry entrepreneur whose tweet was spread by a Kremlin-backed Twitter account, and eventually seen by millions and promoted by the soon-to-be president.

My favorite kind of stories are those that reveal the real people behind influential, anonymous social media accounts. These secret accounts are less reliant on the algorithm, and more carefully crafted to be an escape from public life. They allow someone to keep tabs on and communicate with family and friends apart from their public account, or to communicate the ideas and opinions that for personal or political reasons, they dare not say out loud.

Journalist Ashley Feinberg is the fairy godmother of these kind of juicy stories, ones that unmask the alt accounts of prominent figures like James Comey or Mitt Romney. Her secret was simply a matter of finding smaller accounts of family members that Comey and Romney would naturally want to follow, and then scrolling through them until she found an account that seemed inauthentic but whose content and friends/followers network matched that of these real people.

Be wary of fake accounts

Each platform has its own personality, search capabilities and usefulness in different news situations. But a word of caution with social media accounts: The same rule of trust but verify applies. Groups of people revel in tricking journalists. Especially in breaking news situations, fake accounts will always be born, many with ominous or threatening posts meant to attract reporters. This fake Instagram account used the name of a mass shooter and was created after a shooting at Saugus High School in California. It gained attention via screenshots on Twitter, but BuzzFeed News later revealed it did not belong to the shooter.

Confirming a social account with the subject, family and friends, law enforcement and/or social media PR are ways to protect yourself from being duped.

Finally, and perhaps the most important note: There’s no one right order in which to complete these steps. Often, I’m led down rabbit holes and have more tabs open than I’m proud of. Creating a system that you can replicate — whether it’s tracking your steps in a Google doc or letting a paid tool like Hunchly monitor as you search — is the key to clarifying connections between people and the lives they lead online, and turning those conclusions into stories.

In August 2010 some colleagues and I organised what we believe was one of the first international ‘data journalism’ conferences, which took place in Amsterdam. At this time there wasn’t a great deal of discussion around this topic and there were only a couple of organizations that were widely known for their work in this area.

The way that media organizations like Guardian and the New York Times handled the large amounts of data released by Wikileaks is one of the major steps that brought the term into prominence. Around that time the term started to enter into more widespread usage, alongside ‘computer-assisted reporting’, to describe how journalists were using data to improve their coverage and to augment in-depth investigations into a given topic.

Speaking to experienced data journalists and journalism scholars on Twitter it seems that one of the earliest formulations of what we now recognise as data journalism was in 2006 by Adrian Holovaty, founder of EveryBlock — an information service which enables users to find out what has been happening in their area, on their block. In his short essay “A fundamental way newspaper sites need to change”, he argues that journalists should publish structured, machine-readable data, alongside the traditional ‘big blob of text’:

For example, say a newspaper has written a story about a local fire. Being able to read that story on a cell phone is fine and dandy. Hooray, technology! But what I really want to be able to do is explore the raw facts of that story, one by one, with layers of attribution, and an infrastructure for comparing the details of the fire — date, time, place, victims, fire station number, distance from fire department, names and years experience of firemen on the scene, time it took for firemen to arrive — with the details of previous fires. And subsequent fires, whenever they happen.

But what makes this distinctive from other forms of journalism which use databases or computers? How — and to what extent — is data journalism different from other forms of journalism from the past?

‘Computer-Assisted Reporting’ and ‘Precision Journalism’

Using data to improve reportage and delivering structured (if not machine readable) information to the public has a long history. Perhaps most immediately relevant to what we now call data journalism is ‘computer-assisted reporting’ or ‘CAR’, which was the first organised, systematic approach to using computers to collect and analyze data to improve the news.

CAR was first used in 1952 by CBS to predict the result of the presidential election. Since the 1960s, (mainly investigative, mainly US-based) journalists, have sought to independently monitor power by analyzing databases of public records with scientific methods. Also known as ‘public service journalism’, advocates of these computer-assisted techniques have sought to reveal trends, debunk popular knowledge and reveal injustices perpetrated by public authorities and private corporations. For example, Philip Meyer tried to debunk received readings of the 1967 riots in Detroit — to show that it was not just less-educated Southerners who were participating. Bill Dedman’s “The Color of Money” stories in the 1980s revealed systemic racial bias in lending policies of major financial institutions. In his “What Went Wrong” Steve Doig sought to analyze the damage patterns from Hurricane Andrew in the early 1990s, to understand the effect of flawed urban development policies and practices. Data-driven reporting has brought valuable public service, and has won journalists famous prizes.

In the early 1970s the term ‘precision journalism’ was coined to describe this type of news-gathering: “the application of social and behavioral science research methods to the practice of journalism.” Precision journalism was envisioned to be practiced in mainstream media institutions by professionals trained in journalism and social sciences. It was born in response to “new journalism”, a form of journalism in which fiction techniques were applied to reporting. Meyer suggests that scientific techniques of data collection and analysis rather than literary techniques are what is needed for journalism to accomplish its search for objectivity and truth.

Precision journalism can be understood as a reaction to some of journalism’s commonly cited inadequacies and weaknesses: dependence on press releases (later described as “churnalism”), bias towards authoritative sources, and so on. These are seen by Meyer as stemming from a lack of application of information science techniques and scientific methods such as polls and public records. As practiced in the 1960s, precision journalism was used to represent marginal groups and their stories. According to Meyer:

Precision journalism was a way to expand the tool kit of the reporter to make topics that were previously inaccessible, or only crudely accessible, subject to journalistic scrutiny. It was especially useful in giving a hearing to minority and dissident groups that were struggling for representation.

An influential article published in the 1980s about the relationship between journalism and social science echoes current discourse around data journalism. The authors, two US journalism professors, suggest that in the 1970s and 1980s the public’s understanding of what news is broadens from a narrower conception of ‘news events’ to ‘situational reporting’, or reporting on social trends. By using databases of — for example — census data or survey data, journalists are able to “move beyond the reporting of specific, isolated events to providing a context which gives them meaning.”

As we might expect, the practise of using data to improve reportage goes back as far as ‘data’ has been around. As Simon Rogers points out, the first example of data journalism at the Guardian dates from 1821. It is a leaked table of schools in Manchester listing the number of students who attended it and the costs per school. According to Rogers this helped to show for the first time the real number of students receiving free education, which was much higher than what official numbers showed.

Another early example in Europe is Florence Nightingale and her key report, ‘Mortality of the British Army’, published in 1858. In her report to the parliament she used graphics to advocate improvements in health services for the British army. The most famous is her ‘coxcomb’, a spiral of sections, each representing deaths per month, which highlighted that the vast majority of deaths were from preventable diseases rather than bullets.

Data journalism and Computer-Assisted Reporting

At the moment there is a “continuity and change” debate going on around the label “data journalism” and its relationship with these previous journalistic practices which employ computational techniques to analyze datasets.

Some argue that there is a difference between CAR and data journalism. They say that CAR is a technique for gathering and analyzing data as a way of enhancing (usually investigative) reportage, whereas data journalism pays attention to the way that data sits within the whole journalistic workflow. In this sense data journalism pays as much  — and sometimes more — attention to the data itself, rather than using data simply as a means to find or enhance stories. Hence we find the Guardian Datablog or the Texas Tribune publishing datasets alongside stories, or even just datasets by themselves for people to analyze and explore.

Another difference is that in the past investigative reporters would suffer from a poverty of information relating to a question they were trying to answer or an issue that they were trying to address. While this is of course still the case, there is also an overwhelming abundance of information that journalists don’t necessarily know what to do with. They don’t know how to get value out of data. A recent example is the Combined Online Information System, the UK’s biggest database of spending information — which was long sought after by transparency advocates, but which baffled and stumped many journalists upon its release. As Philip Meyer recently wrote to me: “When information was scarce, most of our efforts were devoted to hunting and gathering. Now that information is abundant, processing is more important.”

On the other hand, some argue that there is no meaningful difference between data journalism and computer-assisted reporting. It is by now common sense that even the most recent media practices have histories, as well as something new in them. Rather than debating whether or not data journalism is completely novel, a more fruitful position would be to consider it as part of a longer tradition, but responding to new circumstances and conditions. Even if there might not be a difference in goals and techniques, the emergence of the label “data journalism” at the beginning of the century indicates a new phase wherein the sheer volume of data that is freely available online combined with sophisticated user-centric tools, self-publishing and crowdsourcing tools enables more people to work with more data more easily than ever before.

Data journalism is about mass data literacy

Digital technologies and the web are fundamentally changing the way information is published. Data journalism is one part in the ecosystem of tools and practices that have sprung up around data sites and services. Quoting and sharing source materials is in the nature of the hyperlink structure of the web and the way we are accustomed to navigate information today. Going further back, the principle that sits at the foundation of the hyperlinked structure of the web is the citation principle used in academic works. Quoting and sharing the source materials and the data behind the story is one of the basic ways in which data journalism can improve journalism, what Wikileaks founder Julian Assange calls “scientific journalism”.

By enabling anyone to drill down into data sources and find information that is relevant to them, as well as to to verify assertions and challenge commonly received assumptions, data journalism effectively represents the mass democratisation of resources, tools, techniques and methodologies that were previously used by specialists — whether investigative reporters, social scientists, statisticians, analysts or other experts. While currently quoting and linking to data sources is particular to data journalism, we are moving towards a world in which data is seamlessly integrated into the fabric of media. Data journalists have an important role in helping to lower the barriers to understanding and interrogating data, and increasing the data literacy of their readers on a mass scale.

At the moment the nascent community of people who called themselves data journalists is largely distinct from the more mature CAR community. Hopefully in the future we will see stronger ties between these two communities, in much the same way that we see new NGOs and citizen media organizations like ProPublica and the Bureau of Investigative Journalism work hand in hand with traditional news media on investigations. While the data journalism community might have more innovative ways delivering data and presenting stories, the deeply analytical and critical approach of the CAR community is something that data journalism could certainly learn from.

Written by Micah Clark

“Fear has big eyes,” goes an old Russian folk saying. “What it sees is what is not there.”

This is a story about fear’s big eyes and the things that were not there.

At approximately 9:50 a.m. on Oct. 22, 2014, Michael Zehaf-Bibeau shot and killed a soldier guarding the Canadian War Memorial in Ottawa. In a scene reminiscent of a Hollywood thriller, Zehaf-Bibeau then charged into the halls of parliament, where he was eventually shot and killed.

Two days earlier, a Canadian soldier was killed when he was deliberately hit by a car driven by a man who had previously drawn the attention of Canadian security agencies. The ensuing shootout on Parliament Hill had Canadians on edge. Was this a terrorist attack? What motivated the attacker? Was ISIS involved?

The speculation reached fever pitch when a photo of the assailant, taken at the very moment of his attack, was posted by a Twitter account claiming affiliation with ISIS. Other Twitter accounts, and eventually Canadian journalists and the Canadian public, rapidly used the photo and the ISIS account that posted it to draw a completely imaginary connection between the assailant and ISIS.

All of this speculation, however, was based on fundamentally incorrect source attribution. The story of the photo’s actual provenance is a remarkable example of the new normal for modern journalism.

The photo was first posted by an unknown user to an Ottawa Police tweet, which asked for any information about the assailant. This occurred sometime before 2 p.m., when Montreal journalist William Reymond located the photo and took a screen capture (Reymond, who has reported extensively on his scoop, has not provided a link to the tweet from Ottawa Police. The time and content he describes suggest it was this tweet). The photo and the account that posted it were deleted almost instantly.

With this exceptional photo in his hands, and to his considerable credit, Reymond took a full two hours to verify its authenticity before posting it to his Twitter account, @Breaking3zero, at 4:16 p.m.

Reymond’s process of verification, which he describes here in detail, included comparing the facial features, clothes and weapon of the man in the photo with surveillance footage, as well as comparing it with details that emerged as witnesses and officials shared details of the attack.

Along with the rifle, two other key pieces of evidence were the fact that the man in the photo was wearing a keffieh, which witness had described, and the fact that he was carrying an umbrella. The shooter used an umbrella to conceal his weapon as he approached the War Memorial, according to reports.

Here is what Reymond tweeted:

It translates to, “After two hours of verification, a source confirmed to me that ‘it looks like the shooter.’ Proceed with caution.”

It was only after Reymond's tweet that an ISIS-related Twitter account, “Islamic Media” (@V_IMS), posted the photo, at approximately 4:45 p.m. This account too has since been suspended and deleted.

“Just twenty minutes after I published it, a French-language feed supporting the Islamic State picks up the photo and posts it,” wrote Reymond. “And that is how some media start to spread the wrong idea that ISIS is at the origin of the photo.”

Within minutes, another Twitter account, @ArmedResearch, posted the photo stating that, “#ISIS Media account posts picture claiming to be Michael Zehaf-Bibeau, dead #OttawaShooting suspect. #Canada.”

In spite of its failure to substantiate this claim or provide appropriate credit to @Breaking3zero, Canadian journalists seized upon @ArmedResearch’s claim, reporting the photo was “tweeted from an ISIS account,” with all the implications that accompany such an assertion.

But as the saying goes, facts are stubborn things. Technical data from @V_IMS’s Twitter page, captured before the account was suspended, show that @V_IMS sourced the photo from @Breaking3zero. The text in grey below shows the original source URL, from twitter.com/Breaking3zero:

The claim that the photo of Zehaf-Bibeau originated with an ISIS account is categorically false. The ISIS account that circulated the photo acquired it hours after it was originally posted to Twitter.

SecDev’s independent monitoring of ISIS’ social media shows that prominent ISIS accounts were reacting to events in Ottawa in much the same way that Ottawans and others were — posting contradictory and often incorrect information about the attack. There is no indication in social media that ISIS had prior knowledge of the attack, or that they were in any way directly affiliated with Zehaf- Bibeau.

Indeed, there is still no evidence to indicate ISIS involvement in the October attack in Ottawa. There is, however, a remarkable photo taken at an incredible moment, a testament to the game-changing power of mobile technology and social media.

The temptation to draw a connection between vivid photos like this one and our worst fears is enormous. Avoiding this temptation is one of the chief responsibilities of 21st century journalists.

Written by Anahi Ayala Iacucci

During the Ukrainian parliamentary elections of fall 2012, Internews Ukraine, a local NGO supported by the global nonprofit media organization Internews, ran an election monitoring project called Elect.UA. It used a mix of crowdsourcing, mobile phones, social media, professional electoral monitoring and media monitoring to oversee the electoral campaign, and possible violations of or tampering with the results.

The project was built upon a fairly complex structure: 36 journalists around the country reported stories during the electoral campaign and on election day. At the same time, three different electoral monitoring organizations had workers reporting to the same platform using SMS, online forms and emails. Elect.UA also invited Ukrainians to report about their election experience using social media (Twitter and Facebook), mobile technology (SMS and a hotline number), a smartphone app, an online form or email.

All information coming from Internews-trained journalists and electoral monitors was automatically tagged as verified, while messages from the crowd were vetted by a team of 16 administrators in Kiev.

For the messages coming from the crowd, the admin team set up a verification protocol based on the source of the information: mobile technology, social media, online form or email.

For each source, the team would try to verify the sender of the information (when possible), the content of the information and the context. For each of those components the team would also try to establish if something could be 100 percent verified, or only partly verified.

For information coming via social media, this image shows the decision tree model used by administrators in the verification process.

The first step was to perform an online search of the information and its source to identify all possible digital traces of that person, and the piece of content. (For example, we examined other social media accounts, mentions by media articles, information about university, affiliations, etc.). The search was aimed at determining if the person was a reliable source, and if there was a trace of the information they provided elsewhere online.

The second step was to use the information collected to build a profile of the person, as well as a profile of the content they provided. For each of the 5Ws - who, what, when, where and why - administrators had to carefully determine what they could prove, and what they could not.

For multimedia content, the source verification protocol was the same, but we had a different path for the content. Photos and video were verified by looking for any identifiable landmarks, and by performing an analysis of the audio (to listen for language, dialects, slang words, background noise, etc.), clothes and of light (artificial or natural), among other ele- ments in the content.

When a piece of information could not be verified with a sufficient degree of certainty, the report was sent back to an electoral monitor or a reporter on the ground for real-time, inperson verification.

For example, on September 28, 2012, Elect.UA received an anonymous message via its website that parliamentary candidate Leonid Datsenko had been invited for a discussion by a stranger, and then was intimidated in order to force him to withdraw from the elections.

The next day, the administrators of the platform found an article in a reliable media source that included a record of the exchange. We still held the report for verification, and then, on October 1, local journalists reported on a press conference about the incident. Elect. UA’s local journalists also conducted interviews with local law enforcement services, who acknowledged this case to be true.

Overall, the Elect.UA team managed to verify an incredible amount of information using these protocols, and also noticed that the more the administrators became familiar with the verification process, the faster they were able to work. This proves that the verification of user generated content is a skill that can be systematized and learned, resulting in efficient, reliable results.

The decision tree model:

Written by Caelainn Barr

Abstract

Building data sets for investigations and powerful storytelling.

Keywords: data journalism, crime, accountability, race, United Kingdom, databases

In early 2017 two colleagues, Gary Younge and Damien Gayle, approached me in The Guardian newsroom. They wanted to examine knife crime in the United Kingdom. While there was no shortage of write-ups detailing the deaths of victims of knife crime, follow-ups on the pursuit of suspects, and reports on the trials and convictions of the perpetrators, no one had looked at all the homicides as a whole.

My first question was, how many children and teenagers had been killed by knives in recent years?

It seemed a straightforward query but once I set out to find the data it soon became apparent—no one could tell me. The data existed, somewhere, but it wasn’t in the public domain. At this stage I had two options, give up or make a data set from scratch based on what I could access, build and verify myself. I decided to build my own data set.

Why Build Your Own Data Set?

Data journalism needn’t be solely based on existing data sets. In fact there is a great case for making your own data. There is a wealth of information in data that is not routinely published or in some cases not even collected.

In building your own data set you create a unique set of information, a one-off source, with which to explore your story. The data and subsequent stories are likely to be exclusive and it can give you a competitive edge to find stories other reporters simply can’t. Unique data sets can also help you identify what trends experts and policy makers haven’t been able to spot.

Data is a source of information in journalism. The basis for using data in journalism is structured thinking. In order to use data to its full potential, at the outset of a project the journalist needs to think structurally: What is the story I want to be able to tell and what do I need to be able to tell it?

The key to successfully building a data set for your story is to have a structured approach to your story and query every source of data with a journalistic sense of curiosity.

Building your own data set encompasses a lot of the vital skills of data journalism—thinking structurally, planned storytelling and finding data in creative ways. It also has a relatively low barrier to entry, as it can be done with or without programming skills. If you can type into a spreadsheet and sort a table, you’re on your way to building the basic skills of data journalism.

That’s not to say data journalism is straightforward. Solid and thorough data projects can be very complex and time-consuming work, but armed with a few key skills you can develop a strong foundation in using data for storytelling.

Building Your Own Data Set Step by Step

Plan what is required. The first step to making or gathering data for your analysis is assessing what is required and if it can be obtained. At the outset of any project it’s worth making a story memo which sketches out what you expect the story will attempt to tell, where you think the data is, how long it will take to find it and where the potential pitfalls are. The memo will help you assess how long the work will take and if the outcome is worth the effort. It can also serve as a something to come back to when you’re in the midst of the work at a later stage.

Think of the top line. At the outset of a data-driven story where the data does not exist you should ask what the top line of the story is. It’s essential to know what the data should contain as this sets the parameters for what questions you can ask of the data. This is essential as the data will only ever answer questions based on what it contains. Therefore, to make a data set that will fulfil your needs, be very clear about what you want to be able to explore and what information you need to explore it.

Where might the data be held? The next step is to think through where the data may be held in any shape or form. One way to do this is to retrace your steps. How do you know there is a potential story here? Where did the idea come from and is there a potential data source behind it?

Research will also help you clarify what exists, so comb through all of the sources of information that refer to the issue of interest and talk to academics, researchers and statisticians who gather or work with the data. This will help you identify shortcomings and possible pitfalls in using the data. It should also spark ideas about other sources and ways of getting the data. All of this preparation before you start to build your data set will be invaluable if you need to work with difficult government agencies or decide to take another approach to gathering the data.

Ethical concerns. In planning and sourcing any story we need to weigh up the ethical concerns and working with data is no different. When building a data set we need to consider if the source and method we’re using to collect the information is the most accurate and complete possible.

This is also the case with analysis—examine the information from multiple angles and don’t torture the data to get it to say something that is not a fair reflection of the reality. In presenting the story be prepared to be transparent about the sourcing, analysis and limitations of the data. All of these considerations will help build a stronger story and develop trust with the reader.

Get the data. Once a potential source has been identified, the next step is to get the data. This may be done manually through data entry into a spreadsheet, transforming information locked in PDFs into structured data you can analyze, procuring documents through a human source or the Freedom of Information Act (FOIA), programming to scrape data from documents or web pages or automating data capture through an application programming interface (API).

Be kind to yourself! Don’t sacrifice simplicity for the sake of it. Seek to find the most straightforward way of getting the information into a data set you can analyze. If possible, make your work process replicable, as this will help you check your work and add to the data set at a later stage, if needed.

In obtaining the data refer back to your story outline and ask, will the data allow me to fully explore this topic? Does it contain the information that might lead to the top lines I’m interested in?

Structure. The key difference between information contained in a stack of text-based paper documents and a data set is structure. Structure and repetition are essential to building a clean data set ready for analysis.

The first step is to familiarize yourself with the information. Ask yourself what the material contains—what will it allow you to say? What won’t you be able to say with the data? Is there another data set you might want to combine the information with? Can you take steps in building this data set which will allow you to combine it with others?

Think of what the data set should look like at the end of the process. Consider the columns or variables you would want to be able to analyze. Look for inspiration in the methodology and structure underlying other similar data sets.

Cast the net wide to begin with, taking account of all the data you could gather and then pare it back by assessing what you need for the story and how long it will take to get it. Make sure the data you collect will compare like with like. Choose a format and stick to it—this will save you time in the end! Also consider the dimensions of the data set you’re creating. Times and dates will allow you to analyze the information over time; geographic information will allow you to possibly plot the data to look for spatial trends.

Keep track of your work and check as you go. Keep notes of the sources you have used to create your data set and always keep a copy of the original documents and data sets. Write up a methodology and a data dictionary to keep track of your sources, how the data has been processed and what each column contains. This will help flag questions and shake out any potential errors as you gather and start to analyze the data.

Assume nothing and check all your findings with further reporting. Don’t hold off talking to experts and statisticians to sense–check your approach and findings. The onus to bulletproof your work is even greater when you have collated the data, so take every step to ensure the data, analysis and write-up are correct.

Case Study: Beyond the Blade

At the beginning of 2017 the data projects team alongside Gary Younge, Damian Gayle and The Guardian’s community journalism team set out to document the death of every child and teenager killed by a knife in the United Kingdom. In order to truly understand the issue and explore the key themes around knife crime the team needed data. We wanted to know—who are the young people dying in the United Kingdom as a result of stabbings? Are they young children or teenagers? What about sex and ethnicity? Where and when are these young people being killed?

After talking to statisticians, police officers and criminologists it became clear that the data existed but it was not public. Trying to piece together an answer to the question would consume much of my work over the next year.

The data I needed was held by the Home Office in a data set called the Homicide Index. The figures were reported to the Home Office by police forces in England and Wales. I had two potential routes to get the information—send a freedom of information request to the Home Office or send requests to every police force. To cover all eventualities, I did both. This would provide us with the historical figures back to 1977.

In order to track deaths in the current year we needed to begin counting the deaths as they happened. As there was no public or centrally collated data we decided to keep track of the information ourselves, through police reports, news clippings, Google Alerts, Facebook and Twitter.

We brainstormed what we wanted to know—name, age and date of the incident were all things we definitely wanted to record. But other aspects of the circumstances of the deaths were not so obvious. We discussed what we thought we already knew about knife crime—it was mostly male with a disproportionate number of Black male victims.

To check our assumptions we added columns for sex and ethnicity. We verified all the figures by checking the details with police forces across the United Kingdom. In some instances this revealed cases we hadn’t picked up and allowed us to cross-check our findings before reporting.

After a number of rejected FOI requests and lengthy delays the data was eventually released by the Home Office. It gave the age, ethnicity and sex of all people killed by knives by police force area for almost 40 years. This, combined with our current data set, allowed us to look at who was being killed and the trend over time.

The data revealed knife crime had killed 39 children and teenagers in England and Wales in 2017, making it one of the worst years for deaths of young people in nearly a decade. The figures raised concerns about a hidden public health crisis amid years of police cuts.

The figures also challenged commonly held assumptions about who knife crime affects. The data showed in England and Wales in the 10 years to 2015, one third of the victims were Black. However, outside the capital, stabbing deaths among young people were not mostly among Black boys, as in the same period less than one in five victims outside London were Black.

Although knife crime was a much-debated topic, the figures were not readily available to politicians and policy makers, prompting questions about how effective policy could be created when the basic details of who knife crime affects were not accessible.

The data provided the basis of our award-winning project which reframed the debate on knife crime. The project would not have been possible without building our own data set.

Now in its 70th year the Australian Broadcasting Corporation is Australia’s national public broadcaster. Annual funding is around AUS$1bn which delivers seven radio networks, 60 local radio stations, three digital television services, a new international television service and an online platform to deliver this ever expanding offering of digital and user generated content. At last count there were in excess of 4,500 full time equivalent staff and nearly 70% of them make content.

We are a national broadcaster fiercely proud of our independence — because although funded by government — we are separated at arm’s length through law. Our traditions are independent public service journalism. The ABC is regarded the most trusted news organzation in the country.

These are exciting times and under a managing director — the former newspaper executive Mark Scott — content makers at the ABC have been encouraged to as the corporate mantra puts it — be ‘agile’.

Of course, that’s easier said than done.

But one initiative in recent times designed to encourage this has been a competitive staff pitch for money to develop multi-platform projects.

This is how the ABC’s first ever data journalism project was conceived.

Sometime early in 2010 I wandered into the pitch session to face with three senior ‘ideas’ people with my proposal.

I’d been chewing it over for some time. Greedily lapping up the data journalism that the now legendary Guardian data journalism blog was offering, and that was just for starters.

It was my argument that no doubt within 5 years the ABC would have its own data journalism unit. It was inevitable, I opined. But the question was how are we going to get there, and who’s going to start.

For those readers unfamiliar with the ABC, think of a vast bureaucracy built up over 70 years. Its primary offering was always radio and television. With the advent of online in the last decade this content offering unfurled into text, stills and a degree of interactivity previously unimagined. The web space was forcing the ABC to rethink how it cut the cake (money) and rethink what kind of cake it was baking (content).

It is of course a work in progress.

But something else was happening with data journalism. Government 2.0 (which as we discovered is largely observed in the breach in Australia) was starting to offer new ways of telling stories that were hitherto buried in the zero’s and dots.

All this I said to the folk during my pitch. I also said we needed to identify new skills sets, train journalists in new tools. We needed a project to hit play.

And they gave me the money.

On the 24th of November 2011 the ABC’s multi-platform project and ABC News Online went live with ‘Coal Seam Gas by the Numbers’.

It was five pages of interactive maps, data visualizations and text.

It wasn’t exclusively data journalism — but a hybrid of journalisms that was born of the mix of people on the team and the story, which to put in context is raging as one of the hottest issues in Australia.

The jewel was an interactive map showing coal seam gas wells and leases in Australia. Users could search by location and switch between modes to show leases or wells. By zooming in users could see who the explorer was, the status of the well and its drill date. Another map showed the location of coal Seam gas activity compared to the location of groundwater systems in Australia.

We had data visualizations which specifically addressed this issue of waste salt and water production that would be produced depending on the scenario that emerged.

Another section of the project investigated the release of chemicals into a local river system

Our team

Where did we get the data from?

The data for the interactive maps were scraped from shapefiles (a common kind of file for geospatial data) downloaded from government websites.

Other data on salt and water were taken from a variety of reports.

The data on chemical releases was taken from environmental permits issued by the government.

What did we learn?

‘Coal Seam Gas by the Numbers’ was an ambitious in content and scale. Uppermost in my mind was what did we learn and how might we do it differently next time?

The data journalism project brought a lot of people into the room who do not normally meet at the ABC. In lay terms — the hacks and the hackers. Many of us did not speak the same language or even appreciate what the other does. Data journalism is disruptive!

The practical things:

The big picture: some ideas

Big media organzations need to engage in capacity building to meet the challenges of data journalism. My hunch is there are a lot of geeks and hackers hiding in media technical departments desperate to get out. So we need ‘hack and hacker meets’ workshops where the secret geeks, younger journalists, web developers and designers come out to play with more experienced journalists for skill sharing and mentoring. Task: download this data set and go for it!

Ipso facto Data journalism is interdisciplinary. Data journalism teams are made of people who would not in the past have worked together. The digital space has blurred the boundaries.

We live in a fractured, distrustful body politic. The business model that formerly delivered professional independent journalism – imperfect as it is — is on the verge of collapse. We ought to ask ourselves — as many now are — what might the world look like without a viable fourth estate? The American journalist and intellectual Walter Lippman remarked in the 1920’s that “it is admitted that a sound public opinion cannot exist without access to news”. That statement is no less true now. In the 21st century everyone’s hanging out in the blogosphere. It’s hard to tell the spinners, liars, dissemblers and vested interest groups from the professional journalists. Pretty much any site or source can be made to look credible, slick and honest. The trustworthy mastheads are dying in the ditch. And in this new space of junk journalism, hyperlinks can endlessly take the reader to other more useless but brilliant looking sources that keep hyperlinking back into the digital hall of mirrors. The technical term for this is: bullshit baffles brains. In the digital space everyone’s a storyteller now — right? Wrong. If professional journalism – and by that I mean those who embrace ethical, balanced, courageous truth seeking storytelling – is to survive then the craft must reassert itself in the digital space. Data journalism is just another tool by which we will navigate the digital space. It’s where we will map, flip, sort, filter, extract and see the story amidst all those 0’s and 1’s. In the future we’ll be working side by side with the hackers, the developers the designers and the coders. It’s a transition that requires serious capacity building. We need news managers who “get” the digital/journalism connection to start investing in the build.

Written by Hamoud Almahmoud

Searching for names of companies or people in the Middle East presents some special challenges. Let us start with a real example I have worked on recently:

I recently received a request from a European reporter who was investigating a company, Josons, which had won a bid to supply weapons in Eastern Europe.

This company was registered in Lebanon. The reporter had come up empty when searching for information in online Lebanese business registries.

I immediately started to think about how this company would be spelled in Arabic, and especially with the Lebanese accent. Of course, I knew beforehand that this company name must be mentioned in English inside the online company records in Lebanon. But the search engine of the Lebanese commercial registry shows results only in Arabic. This was why the reporter had come up empty.

For example, a search for “Josons” in the official Commercial Register gives us this result:

As you can see, the results are (0), however, we should not give up and quit. The first step is to guess how Josons is written in Arabic. There could be a number of potential spellings. To start, I put did a Google search with the word

“Lebanon” in Arabic next to the English company name: josons !لبنا. The first page of search results shows that the company’s Arabic name is جوسانز as in this official directory:

That was also confirmed by searching in an online Lebanese business directory.

Now we have the company name in Arabic. A search with the name جوسانز in the Commercial Register

shows that the company was registered twice — once onshore and another offshore.

Cultures of writing

That was one example of how to deal with language challenges when gathering information about companies in the MENA region. Doing this work often requires working with Arabic, French, English and Kurdish, in addition to many different Arabic accents.

The first step is to determine which language to search for the information you need, and then to figure out the spelling in Arabic. However, keep in mind that the pronunciation of a single word can differ widely among Arabic- speaking countries.

For example, in order to search for a holding company, it’s useful to know how to write the word “group” in the Arabic database of business registries. However, there are three different ways of writing this word based on how the English word “group” is transliterated into Arabic. (Arabic has no letter for the “p” sound.).

1. In the Jordanian business registry, for example, it is written as:

2. In Lebanon, its:

3. The third spelling is shown in the Tunisian registry of commence:

Also be aware that even within the same registry you should search using multiple spellings of the same word. For example, the word “global” might be written like 0غلوبا or like غلوبل. You can find both spellings in the Bahraini business registry:

These examples demonstrate how an understanding of cultures, languages and other factors can play a role in ensuring how effectively you can make use of public data and information during an investigation.

Abstract

Numbers are seemingly uncomplicated and straightforward measures of value, but beware—numbers hide moral and political trouble.

Keywords: ecosystems value, Australian environmental governance, sociology of quantification, data journalism, science and technology studies

At the turn of the century the Australian state developed an environmental policy that saw it fully subsidizing labour costs incurred by landowners if they undertook specifically agreed upon landscape work that was designed to reverse environmental degradation.

However, given the almost total domination by neoliberal ideologues in this policy area at that time, the policy was described in the dizzying double-talk of value of ecosystems services. In policy documents it was described as “purchasing environmental interventions to enhance the state’s environmental value.”

Thus in 2009 a state government department would make this almost incomprehensible claim about the success of this policy: In 2009 the contribution to Australia’s GDP from transactions in which the state purchased environmental interventions to enhance ecosystems value from rural landholders in the Corangamite Natural Resource Management Region (NRMR) was calculated as AUD4.94 million.

The number that I narrate here emerged in a press statement issued by the government of the Australian state of Victoria in 2009.

The media release announced the success of investment by the state government in environmental conservation in one of Australia’s 57 NRMRs.

The environmental administrative region of grassy basalt plains that spreads east–west in south-central Victoria is named Corangamite, an Aboriginal term that replaced a name bestowed by the first British pastoralists who in the mid-19th century invaded this country from Tasmania.

They called the region “Australia Felix” and set about cutting down all the trees. The squatters, who subsequently became landowners here, would in less than a century become a sort of colonial landed gentry.

In 2008, in operating the EcoTender Programme in the Corangamite NRMR, the Victorian government purchased ecosystems services value from the descendants of those squatters in pay-as-bid auctions. In 2009 the contribution to Australia’s GDP from these transactions was calculated as AUD4.94 million. The announcement of this value was the occasion of the media release where I first met the number.

I doubt that any journalists picked up on the news promulgated in this brief, including its numbered value; this number is hardly hot news.

In the context of a press release the naming of a specific number value reassures. The national accounts are important and real, and if this regional government intervention features as a specified value contributing to the national economy, then clearly the government intervention is a good thing.

The specification of value here claims a realness for the improvements that the government interventions are having. The implication is that this policy leads to good environmental governance. Of course, the actual value the number name (AUD4.94 million) points to, what it implicitly claims to index, is not of much interest to anyone. That a number appears to correspond to something “out there” that can be valued, is good enough for purposes of reassuring.

My narration of this number offers a mind-numbingly detailed account of the sociotechnical means by which the number came to life. The story has the disturbing effect of revealing that this banal number in its workaday media release is a paper-thin cover-up. Profound troubles lurk.

Before I begin to tell my story and articulate the nature of these profound troubles that seem to shadow any doing of valuation, even such a banal doing, let me pre-emptively respond to some questions that I imagine might be beginning to emerge for readers of The Data Journalism Handbook.

First, I acknowledge that telling a story of how a number has come to life rather than finding some means to promote visualization of what that number means in a particular context, is rather an unusual approach in contemporary data journalism. I can imagine a data journalist doubting that such storytelling would work. Perhaps a first response is to remind you that it is not an either/or choice and that working by intertwining narrative and visualizing resources in decoding and interpreting is an effective way to get ideas across. In presenting such an intertwining, journalists should always remember that there are two basic speaking positions in mixing narratives and visuals.

One might proceed as if the visual is embedded within the narrative, in which case you are speaking to the visual, which seems to represent or illustrate something in the story. Or, you can proceed as if the narrative is embedded in the visual, in which case you are speaking from within, diagram. This is a less common strategy in data journalism, yet I can imagine that the story I tell here could well be used in that way. Of course, switching between these speaking positions within a single piece is perhaps the most effective strategy (for an account of such switching, see Verran & Winthereik, 2016).

Second, you might see it as odd to tell a story of a very particular number when what clearly has agency when it comes to decision–making and policy design, and what data journalists are interested in, is what can be made of data sets in mobilizing this algorithm or that. This worry might prompt you to ask about relations between numbers and data sets.

The answer to such a query is fairly straightforward and not very interesting. There are many numbers in a data set; the relation is a one–many relation albeit that numbers are assembled in very precise arrays. The more interesting question enquires about the relation between numbers and algorithms.

My answer would be that while algorithms mobilize a protocol that elaborates how to work relations embedded in a database, numbers express a protocol that lays out how to work relations of collective being. Numbering is a form of algorithming and vice versa.1

We could say that numbers are to algorithms as a seed is to the plant that might germinate from it; to mix metaphors, they have a chicken-and-egg relation. While there are certain interestingly different sociotechnical characteristics of generating enumerated value by analogue means (mixing cognitive, linguistic and graphic resources), of conventional enumeration as taught to primary school children, and of contriving enumerated value by digital computation, it is the sameness that matters here: AUD4.94 million has been generated algorithmically and expresses a particular set of relations embedded in a particular data set, but it still presents as just a number.2

So now, to turn to my story. The intimate account of number making I tell here as a story would enable a journalist to recognize that the good news story that the government is slyly soliciting with its media release is not a straightforward matter.

We see that perhaps a political exposé would be more appropriate. The details of how the number is made reveal that this public–private partnership environmental intervention programme involves the state paying very rich landowners to do work that will increase the value of their own property.

The question my story might precipitate is, how could a journalist either celebrate or expose this number in good faith? When I finish the story, I will suggest that that is not the right question.

Narrating a Number

What is the series of sociotechnical processes by which ecosystems services value comes into existence in this public-private partnership programme in order that this value might be traded between government as buyer and landowner as vendor? And exactly how does the economic value of the trade come to contribute to the total marginal gains achieved in the totality of Australian economic activity, Australia’s gross domestic product (GDP)?

I attend to this double-barrelled question with a step-by-step laying out of what is required for a landholder to create a product—“ecosystems services value”—that can compete in a government-organized auction for a contract to supply the government with ecosystem services value.

The messy work in which this product comes to life involves mucking around in the dirt, planting tree seedlings, fixing fences, and generally attempting to repair the damage done to the land perhaps by the landowner’s grandparents, who heedlessly and greedily denuded the country of trees and seeded it with water-hungry plants, in hopes of more grain or more wool and family fortune. Ecosystems services value is generated by intervening in environmental processes.

The value, which is the product to be traded, begins in the work of public servants employed by a Victorian state government department (at that time the Department of Sustainability and Environment, DSE). Collectively these officials decide the areas of the state within which the administration will “run” tenders. In doing this, EnSym, an environmental systems modelling platform, is a crucial tool. This computing capacity is a marvel; it knows “nature out there” as no scientist has ever known nature. Precise and focused representations can be produced—probably overnight.

This software has been developed by the ecoMarkets team and incorporates science, standards, metrics and information developed within DSE, as well as many leading international and national scientific models. EnSym contains three main tools—the “Site Assessment Tool'' for field work, the “Landscape Preference Tool” for asset prioritisation and metric building, and “BioSim'' for catchment planning. (DSE, 2018)

Prioritizing and mapping the areas of the state where auctions will be established, specifying and quantifying the environmental benefits, the ecological values, that might be enhanced through on-ground conservation and revegetation works, are recorded in numerical form. They represent ecosystem properties in the “out there” land.

And the computer program can do more than that, it can also produce a script for intervention by humans. Just as the script of a play calls for production, so too does this script. And, as that script comes to life, “nature out there” seems to draw closer. It ceases to be an entirely removed “nature out there” and becomes nature as an infrastructure of human lives, an infrastructure that we might poke around in so as to fix the “plumbing.”

When the script for a choreographed production of collective human effort is ready, the government calls for expressions of interest from landholders in the project area as the next step. In response to submitted expressions of interest, a government officer visits all properties. We can imagine this officer as taking the general script generated by EnSym along to an actual place at a given time. He or she has a formidable translation task ahead.

The field officer assesses possible sites for works that might become a stage for the production of the script. The aim is to enhance the generation of the specified ecosystems services, so the officer needs to assess the likelihood that specified actions in a particular place will produce an increase in services provision from the ecosystem, thus increasing the value of that particular ecosystems service generated by that property, and through adding together the many such increases generated in this intervention programme, by the state as a whole.

Together the landowner and the government officer hatch a plan. In ongoing negotiation, a formalized management plan for specified plots is devised. The field officer develops this plan in contractable terms. Landholders specify in detail the actual work they will do to carry out the plan. Thus, a product that takes the form of a particular “ecosystems services value” is designed and specified as a series of specified tasks to be completed in a specified time period: So many seedlings of this set of species, planted in this array, in this particular corner of this particular paddock, and fenced off to effect a conservation plot of such and such dimensions, using these materials.

Landholders calculate the cost of the works specified by the state, no doubt including a generous labour payment. They come up with a price the government must pay if it is to buy this product, a particular “ecosystems services value.” Here they are specifying the amount of money they are willing accept to undertake the specified works and hence deliver the ecosystems services value by the specified date. They submit relevant documents to the government in a sealed envelope.

So how does the subsequent auction work? Here EnSym becomes significant again in assessing the bids. Not only a knower of “nature out there,” and a writer of scripts for intervention in that “out there” imagined as infrastructure, EnSym is also a removed judging observer that can evaluate the bids that have been made to produce that script, much like a Warner Bros. might evaluate competing bids to produce a movie. Bids are ranked according to a calculated “environmental benefits index” and the price proposed by the landowner. We must suppose that the government buys the product which offers the highest “environmental benefits index” per unit cost.

Bid assessment. All bids are assessed objectively on the basis of the estimated change in environmental outcomes; the value of the change in environmental outcomes; the value of the assets affected by these changes (significance); dollar cost (price determined by the landholder). (DSE, 2018)

When the results of the auction are announced, selected bidders sign a final agreement based on the management plan and submitted schedule of works, as defined spatial and temporal organization. When all documents are signed, reporting arrangements are implemented and payment can begin: “DSE forwards payment to signed-up landholders on receipt of an invoice. Payments occur subject to satisfactory progress against actions as specified in the Management Agreement” (DSE, 2018).

This Is a Good Thing, Right?

What I have laid out is a precise description of how to buy and sell ecosystems services value. This takes me back to the press release. A quick reading of the media statement might leave a reader with the impression that AUD4.94 million is the value of the additional natural capital value that this govern- ment programme has generated. At first glance AUD4.94 million appears to be the marginal gain in Australia’s natural capital value that was achieved in the programme. But that is a mistake. AUD4.94 million is not the name of a natural capital value. I explain what this number name references below. At this point I want to stay with the product that has been bought and sold in this auction. This product is the trouble I want to stay with.

I want to ask about the value of the increase in “ecosystems services value” that this elaborate and rather costly government programme has achieved. A careful reading of the details of the work by which this increase in value comes into being reveals that nowhere and at no time in the process has that value ever been named or specified. The product that is so rigorously bought and sold is an absence. And worse, there is literally no way that it could ever be otherwise. The programme is a very elaborate accounting exercise for a means of giving away money. When this becomes clear to an outsider, it also becomes obvious that this actuality of what the exercise is has never been hidden. When it comes down to it, this programme is a legitimate means for shifting money from the state coffers into the hands of private landowners.

Recognizing that this is a programme of environmental governance in a liberal parliamentary democracy in which the social technology of the political party is crucial, let me as your narrator temporarily put on a party-political hat. Corangamite is an electorate that has a history of swinging between choosing a member of the left-of-centre party (Labour Party) or a member of the right-of-centre party (Liberal Party) to represent the people of the area in the Victorian Parliament. It is clearly in the interests of any government—left-leaning or right-leaning—to appeal to the voters of the electorate.

And there is no better way to do that than by finding ways to legitimately transfer resources from the state to the bank accounts of constituents. That there is no possibility of putting a number on the value of the product the state buys and the landowners sell here, is, on this reading, of no concern.

So, let me sum up. Economically this programme is justified as generating environmental services value. Described in this way this is a good news story. Taxpayer money used well to improve the environment and get trees planted to ameliorate Victoria’s excessive carbon dioxide generation. Problematically the increase in the value of Victoria’s natural capital cannot be named, articulated as a number, despite it being a product that is bought and sold. It seems that while there are still technical hitches, clearly, this is a good thing.

But equally, using a different economics this programme can just as legitimately be described as funding the labour of tree planting to enhance property values of private landowners. It is a means of intervening to put right damage caused by previous government programmes subsidizing the misallocated labour of land clearing that in all likelihood the landowner’s grandparents profited by, creating a benefit which the landowner continues to enjoy. On this reading the government policy effected in EcoTender is an expensive programme to legitimately give away taxpayer money. Clearly, this is a bad thing.

On Not Disrespecting Numbers and Algorithms: Staying With the Troubles of Value

So, what is a journalist to do? Writing as a scholar and not as a journalist, I can respond to that obvious question only vaguely. In the beginning I return to my claim that the number name used in the press release is a paper-thin cover-up to divert attention from lurking trouble. As I see it, valuation always brings moral trouble that can never be contained for long. The right question to ask I think is, “How might a data journalist respond to that moral trouble?” First, I clear up the matter of the AUD4.94 million. What is this figure?

Where does this neatly named monetary value come from? This is how it is described in an academic paper offering critical commentary on the EcoTender programme:

Under this market-based model economic value from ecosystems services is created when the per-unit costs of complying with the conservation contract are less than the per-unit price awarded to the successful participants in the auction. While [for these sellers] some economic value is lost through the possibility of foregone production of marketed commodities, the participation constraint of rational landowners ensures that there will be a net increase in [economic] value created in the conduct of the auction. (G. Stoneham et al., 2012)

Under the economic modelling of this policy, the assumption is that land- owners will efficiently calculate the costs they will incur in producing the government’s script for intervening in nature as infrastructure—in generating a more efficient performance of the workings of natural infrastructure. Everyone assumes that a profit will be made by the landowner, although, of course, it is always possible that instead of a profit the landowner will have miscalculated and made a loss, but that is of no interest to the government as the buyer of the value generated by the landowners’ labour.

What is of interest to the government is the issue of how this economic transaction can be articulated in a seemly manner.

This is quite a problem when the product bought and sold has an existence solely within the circuit of an auction. The solution to this problematic form of being of the product is the elaborate, complex and complicated technology of the national accounts system. Establishing a market for ecosystems services value, the government wants to show itself as making a difference in nature. And the national accounts are the very convenient place where this can be shown in monetary terms. The “environmental benefits index,” the particular value on the basis of which the government has purchased a particular product—an environmental services value—is ephemeral. It exists solely as a flash, a moment in the auction (Roffe, 2015).

Despite this difficulty in the form of its existence, by ingenious contrivance, both the means of buying and selling something that has a single ephemeral moment of existence is achieved, and evidence of the specific instance of economic activity can be incorporated into the national accounts, albeit that some economists have serious reservations about accuracy (G. Stoneham et al., 2012).

AUD4.94 million is remote from the action of the EcoTender programme and from the nature it is designed to improve. But clearly, if the government makes a statement that its programmes have successfully improved a degraded and damaged nature it is best to find a way to indicate the extent of that improvement. It seems any number is better than none in this situation. And certainly, this is a happy, positive number. An unhappy, negative number that no doubt is available to the government accountants—the value of the cost of running the government programme—would never do here.

Why go on about this oddly out of place number name? Surely this is going a bit far? What is the harm of a little sleight of hand that is relatively easily picked up? My worry here is that this is a misuse of a number that seems to be deliberate. It fails to respect numbers, and refuses to acknowledge the trouble that numbering, or in this case algorithming, always precipitates. It trashes a protocol.

My narrating of a number I found on a visit to a government website has unambiguously revealed a government programme that generates social goods and bads simultaneously. The sleight of hand number naming (using the precise value AUD4.94 million in the media release) that I also found in my narration, points off to the side, at something that is always threatening to overwhelm us: Valuation as a site of moral tension and trouble.

Is the big claim here that value is moral trouble that can never be contained for long? Value theory is a vast topic that has ancient roots in all philosophical traditions, and this is a rabbit warren of vast proportions that I decline to enter. I merely note that claims, often heard over the past 30 years, that the invisible hand of the market tames the moral trouble that tracks with value, are a dangerous exaggeration. Markets might f ind ways to momentarily and ephemerally tame value—as my story reveals. But the trouble with value always returns.

Attending to that is the calling of the data journalist.

Here are a few suggestions on how a data journalist might respect numbers and algorithms—as protocols.

When you are faced with an untroubled surface, where no hint of moral tension is to be found, but still something lurks, then “prick up” your ears and eyes.

-Attune yourself to numbers and algorithms in situ; work out how to think with a number that catches at you.

-Find ways to dilate the peepholes that number names cover.

-Cultivate respectful forms of address for numbers and algorithms in practicing curiosity in disciplined ways.

-Recognize that numbers have pre-established natures and special abilities that emerge in encounter; that the actualities of series of practices by which they come to be, matter.

-Be sure that when you can do these well enough, surprises lie in store. Interesting things happen inside numbers as they come to be.

Footnotes

1. The idea that numbers and algorithms have a sameness is possibly new for many readers, so used are they to thinking of numbers as “abstractions.” My (unusual) account of numbers has them as very ordinary material semiotic entities that inhabit the here and now. For an account of differing protocols mobilizing relations within a single moment of collective being, see Watson, H. (1990). Investigating the social foundations of mathematics: Natural number in culturally diverse forms of life.Social Studies of Science, 20(2), 283–312. doi.org/10.1177/030631290020002004, or Verran, H. (2001). Two consistent logics of numbering. In Science and an African logic (pp. 177–205). University of Chicago Press.

2. For an account of differing sociotechnical characteristics of three numbers that variously emerge in analogue or digital environments, see Verran, H. (2015). Enumerated entities in public policy and governance. In E. Davis & P. J. Davis (Eds.),Mathematics, substance and surmise: Views on the meaning and ontology of mathematics (pp. 365–379). Springer International Publishing. doi.org/10.1007/978-3-319-21473-3_18

Works Cited

DSE. (2018). Innovative market approaches: EcoMarkets. EcoTender and BushT- ender. Department of Sustainability and Environment [now Department of Environment, Land, Water and Planning (DELWP)], Victoria State Government, Australia. www.environment.vic.gov.au/innovative-market-approaches

Roffe, J. (2015). Abstract market theory. Palgrave Macmillan UK.

Stoneham, G., O’Keefe, A., Eigenraam, M., & Bain, D. (2012). Creating physical environmental asset accounts from markets for ecosystem conservation. Ecological Economics, 82, 114–122. doi.org/10.1016/j.ecolecon.2012.06.017

Verran, H., & Winthereik, B. R. (2016). Innovation with words and visuals. A baroque sensibility. In J. Law & E. Ruppert (Eds.), Modes of knowing (pp. 197–223). Mattering Press.

The term ‘data journalism’ can cover a range of disciplines and is used in varying ways in news organizations, so it may be helpful to define what we mean by ‘data journalism’ at the BBC. Broadly the term covers projects that use data to do one or more of the following:

These categories may overlap and in an online environment can often benefit from some level of visualization.

Make It Personal

On the BBC News website we have been using data to provide services and tools for our users for well over a decade.

The most consistent example, which we first published in 1999, is our school league tables, which use the data published annually by the government. Readers can find local schools by entering a postcode, and compare them on a range of indicators. Education journalists also work with the development team to trawl the data for stories ahead of publication.

When we started to do this there was no official site that provided a way for the public to interrogate the data. But now that the Department for Education has its own comparable service, our offering has shifted to focus more on the stories emerging from the data.

The challenge in this area must be to provide access to data in which there is a clear public interest. A recent example of a project where we exposed a large dataset not normally available to the wider public was the special report Every death on every road. We provided a postcode search allowing users to find the location of all road fatalities in the UK in the past decade.

We visualized some of the main facts and figures emerging from the police data and, to give the project a more dynamic feel and a human face, we teamed up with the London Ambulance Association and BBC London radio and TV to track crashes across the capital as they happened. This was reported live online, as well as via Twitter using the hashtag #crash24 and the collisions were mapped as they were reported.

Simple Tools

As well as providing ways to explore large data sets, we have also had success creating simple tools for users that provide personally relevant snippets of information. These tools appeal to the time-poor who may not choose to explore lengthy analysis. The ability to easily share a ‘personal’ fact is something we have begun to incorporate as standard.

A light-hearted example of this approach is our feature The world at 7 billion: What’s your number? published to coincide with the official date at which the world’s population exceeded 7 billion. By entering their birth date the user could find out what ‘number’ they were, in terms of the global population, when they were born and then share that number via twitter or Facebook. The application used data provided by the UN population development fund. It was very popular, and became the most shared link on Facebook in the UK in 2011.

Another recent example is the BBC budget calculator which enabled users to find out how better or worse off they will be when the Chancellor’s budget takes effect — and then share that figure. We teamed up with the accountancy firm KPMG LLP, who provided us with calculations based on the annual budget, and then we worked hard to create an appealing interface that would encourage users to complete the task.

Mining The Data

But where is the journalism in all this? Finding stories in data is a more traditional definition of data journalism. Is there an exclusive buried in the database? Are the figures accurate? Do they prove or disprove a problem? These are all questions a data journalist or computer assisted reporter must ask themselves. But a great deal of time can be taken up sifting through a massive data set in the hope of finding something remarkable.

In this area we have found it most productive to partner with investigative teams or programs which have the expertise and time to investigate a story. The BBC current affairs program Panorama spent months working with the Centre for Investigative Journalism, gathering data on public sector pay. The result was a TV documentary and online the special report Public Sector pay: The numbers where all the data was published and visualized with sector by sector analysis.

As well as partnering with investigative journalists, having access to numerate journalists with specialist knowledge is essential. When a business colleague on the team analyzed the spending review cuts data put out by the government he came to the conclusion that it was making them sound bigger than they actually were. The result was an exclusive story, Making sense of the data, complemented by a clear visualization which won a Royal Statistical Society award.

Understanding An Issue

But data journalism doesn’t have to be an exclusive no-one else has spotted. The job of the data visualization team is to combine great design with a clear editorial narrative to provide a compelling experience for the user. Engaging visualizations of the right data can be used to give a better understanding of an issue or story and we frequently use this approach in our story-telling at the BBC. Heat-mapping data over time to give clear view of change is one technique used here in our UK claimant count tracker.

The data feature Eurozone debt web explores the tangled web of intra-country lending. It helps to explain a complicated issue in a visual way, using colour and proportional arrows combined with clear text. An important consideration is to encourage the user to explore the feature, or follow a narrative, and never feel overwhelmed by the numbers.

Team Overview

The team that produces data journalism for the BBC News website is comprised of about 20 journalists, designers and developers.

As well as data projects and visualizations the team produces all the infographics and interactive multimedia features on the news website. Together these form a collection of story-telling techniques we have come to call ‘visual journalism’. We don’t have people who are specifically identified as ‘data’ journalists, but all editorial staff on the team have to be proficient at using basic spreadsheet applications such as Excel and Google Docs to analyze data.

Central to any data projects are the technical skills and advice of our developers and the visualization skills of our designers. While we are all either a journalist, designer or developer ‘first’ we continue to work hard to increase our understanding and proficiency in each other’s areas of expertise.

The core products for interrogating data are Excel, Google Docs and Fusion Tables. The team has also, but to a lesser extent, used MySQL and Access databases and Solr for interrogating larger data sets and used RDF and SPARQL to begin looking at ways in which we can model events using Linked Data technologies. Developers will also use their programming language of choice, whether that’s ActionScript, Python or Perl, to match, parse or generally pick apart a dataset we might be working on. Perl is used for some of the publishing.

We use Google and Bing Maps and Google Earth along with Esri’s ArcMAP for exploring and visualizing geographical data.

For graphics we use the Adobe Suite including After Effects, Illustrator, Photoshop and Flash, although we would rarely publish Flash files on the site these days as JavaScript, particularly JQuery and other JavaScript libraries like Highcharts, Raphael and D3 increasingly meet our data visualization requirements.

Written by Trushar Barot

One powerful image can define a story.

That was certainly the case for BBC News’ User Generated Content hub in the beginning of July 2005. It had been one week since the initial pilot team was set up to help collate the content being sent to BBC News by its audiences, and help get the best of it shown across TV, radio and online.

Then the July 7 bombings in London happened.

That morning, as the BBC and other news organizations reported a power surge on the London Underground, the UGC team started seeing a very different story emerging via content sent to BBC News directly from its audience.

This was one of the first images the team received. Before it was broadcast, the image was examined closely and the originator was contacted to verify his story and the details of what he saw. The photo inadvertently became one of the first examples of the UGC image verification process that has since moved toward standard practice across the industry.

That image, and others like it, showed the terror and chaos in London during the moments immediately after the attacks. As a result, it ensured that the reporting of the story quickly changed. It was the first significant example of UGC’s proving critical to helping BBC News tell a major story more accurately, better and faster.

Today, the UGC team is embedded within the heart of the BBC newsroom. Its 20 journalists work across TV, radio, online and social media platforms to produce content sourced either directly from the BBC’s audiences or from the wider Web.

Verification is critical to the success of what the UGC team produces. Technology has moved on considerably since 2005, bringing an exponential rise in the use of social networks and the power of mobile phones. These changes offer great benefits in our newsgathering processes, particularly on breaking news; they also bring great challenges.

Whether a trusted global news organization like the BBC or a humanitarian professional on the ground, the need to be fast at collecting and disseminating key images on a breaking news story has to be balanced with the need to be sure the images are credible and genuine. We also have to ensure copyright is protected and appropriate permissions are sought.

Since that day in 2005, the UGC team has developed a number of approaches to help in this process. While the technology will continue to change - as will the tools we use - the basic principles of image verification remain the same:

1. Establish the author/originator of the image.
2. Corroborate the location, date and approximate time the image was taken.
3. Confirm the image is what it is labeled/suggested to be showing.
4. Obtain permission from the author/originator to use the image.

Let’s look at these points in more detail.

1. Establish the author/originator of the image

The obvious - and usually most effective - way of doing this is to contact the uploader and ask him directly if he is indeed the person who took the image.

Reaching out to the uploader via the social network account or email address the image was shared from is a first step, but it’s also important to try to ascertain as much about the uploader’s identity as possible. These details can help in determining whether he is in fact the original source of the image.

As outlined in the previous chapter, in many instances, people may try to be helpful by repost- ing images they have seen elsewhere. This happens frequently to news organizations - images are sent in by well-meaning members of the public to help report a story. Just by asking the sender to confirm if it’s his image or not can save a lot of time in the verification process.

While tracking down the source of an image begins with the person who uploaded it, it often ends with a different person – the one who actually captured the image.

As referenced in an earlier chapter, an important step is to use a service like Google Reverse Image Search or TinEye. Paste the image URL or a copy of the image into either and they will scan the web to see if there are any matches. If several links to the same image pop up, click on “view other sizes” to investigate further.

Usually, the image with the highest resolution/size should take you to the original source. (On Google Images, the resolution for each image result is listed just next to the image itself.) You can then check it against the image you have and see if the source appears authentic.

Quite often on a breaking news event, there will be no images of specific people that you want to illustrate the story with (particularly if they involve ordinary members of the public). Alternatively, you might want to confirm that an image you have of someone is actually them and not someone else with the same name.

I’ve found Pipl.com to be particularly helpful here as it allows you to cross-reference names, usernames, email address and phone numbers against online profiles of people. For interna- tional searches, WebMii is an additional resource that can help. LinkedIn is also proving to be a great way of verifying individuals and often provides additional leads for being able to track them down (through companies/organizations they are currently or previously associated with).

2. Corroborate the location, date and approximate time the image was taken

There are some useful journalistic and technical ways of establishing information such as date, location and other important details. One core way of gathering this information is when you speak to the creator/uploader of the image. These five questions continue to stand the test of time:

• Who are they?
• Where are they?
• When did they get there?
• What can they see (and what does their photo show)?
• Why are they there?
  

One important aspect to note here: If the image is from a dangerous location, always check that the person you are talking to is safe to speak to you. Also be aware of any issues about identifying the source through any details you broadcast about him or his images.

From our experience at the BBC, people who were really there will give visual answers, often describing the details in the present tense. (“I’m in the middle of X Street; I can see and hear Y.”) The more vague the answer, the more caution you should exercise about what the source is telling you.

Another useful technique is to ask the person to send any additional images shot at the same time. It’s rare that someone takes only one picture in a newsworthy situation. Having more than one image helps you learn more about how the events in question unfolded.

Once you’ve gathered the source’s account of how the image was taken, work to corroborate the information further. Two primary methods can be used to investigate the contents of the photo itself and triangulate that with what you were told by the source.

First, check if the image has any metadata. Metadata, also referred to as “EXIF” data when it comes to digital images, refers to information embedded in an image. If the image is an original, there’s a good chance you will see information about the make and model of the camera, the timestamp of the image (be careful though - if there is one, it could still be set to the manufacturer’s factory setting or another time zone), and the dimensions of the original image, among other details. You can use software like Photoshop (look at the file informa- tion) or look for or free online tools like Fotoforensics.com or Findexif.com to generate an EXIF report.

Upload the image and the EXIF reader will return out whatever information is contained on the image. Some of the information is useful to those who have a more technical understanding of digital photography. But for the average person, data such as the date the photo was originally taken or the type of camera that took the image can sometimes help expose a lying source.

One note of caution here: The majority of social media image sites such as Twitter, Facebook and Instagram strip out most of the original metadata from images when they are uploaded onto their platforms, if not all. (Flickr seems to be an exception to this.)

Second, cross-reference the image with other sources. Awaken your inner investigator by examining the image closely. Quite often there will be clues that can help you verify the location and time it was taken:

• License/number plates on vehicles
• Weather conditions
• Landmarks
• Type of clothing
• Signage/lettering
• Is there an identifiable shop or building?
• What is the type of terrain/environment in the shot?

3. Confirm the image is what it is labeled/suggested to be showing

An image may be authentic, but it could be inaccurately labeled. For example, during Hurricane Sandy, this image spread widely on Twitter and was described as being a shot of three soldiers standing guard at the Tomb of the Unknown Soldier during the storm:

The image was accurate in that it did show soldiers at the Tomb. But it had been taken a month earlier, not during Sandy. The picture had been posted on the Facebook page of the First Army Division East.

As part of verifying the date, time and approximate location of an image, it’s also important you confirm that the image is what it purports to be. An authentic image can still be placed in a false context.

Use Google Maps, Bing Maps or Wikimapia to help you verify locations. UGC images are increasingly being tagged on these services now, and they can also provide useful leads to follow up on, as well as different angles to locations you are investigating. (Learn more about using these mapping services for verification in Chapter 5: Verifying Video.)

Use weather sites that can give you accurate reports of conditions at that location on that date to confirm if the weather in the image matched. As noted in the previous chapter, Wolfram Alpha is very good at searching for weather reports at specific times and places.

If there is lettering (e.g. on a sign) in a different language within the image, use Google Translate to see if it can give you another clue to the location. The optical character reading tool free-ocr.com can also be helpful if you want to extract text from an image -which you can then run through an online translation.

Social media location services like Geofeedia and Ban.jo can also help establish the loca- tion from which an image was uploaded. These services use the GPS data from the mobile device that uploaded the image. While they currently capture only a small percentage of the social media content uploaded from a given location, they do provide a useful initial filter. The image below is an example of some of the photos captured by Geofeedia in the immedi- ate aftermath of the Boston marathon bombings:

Along with those tools and techniques, for images it’s also useful to check to see if similar images are being distributed by official news organizations or agencies. Are there any images from that location being uploaded on social media by others? If they show a similar scene from a different angle, that will also help establish credibility of the image.

Finally, on a big story, it’s always worth double checking if a particularly strong image you come across appears on Snopes, which specializes in debunking urban legends and misin- formation on the Internet.

4. Obtain permission from the author/originator for use of the image

It is always best practice to seek permission from the copyright holder of images. Adding to this, copyright laws in many countries are increasingly clear that damages can be sought by the originator if permission isn’t asked for or granted.

The terms and conditions with regards to the copyright of content uploaded on social media sites vary from service to service. Some, like Flickr, show clearly alongside the image if the photographer has retained all copyright, or if he allows Creative Commons usage. (It’s a good idea to read up on Creative Commons licenses so you are familiar with how they differ.)

When seeking permission, it’s important to keep a few details in mind:

• Be clear about the image(s) you wish to use.
• Explain how the image(s) will be used.
• Clarify how the photographer wishes to be credited (name, username, etc., keeping in mind that in some cases they may wish to remain anonymous).

Most importantly, remember that if you’ve gone through the above checks and processes and you’re still in doubt - don’t use the image!

Written by Tahu Kukutai and Maggie Walter

Abstract

This chapter discusses some of the potential harms of digitalization and considers how Indigenous data sovereignty (ID-SOV), as an emerging site of science and activism, can mediate risks while providing pathways to benefit.

Keywords: Indigenous data sovereignty, activism, data journalism, statistical surveillance, Indigenous peoples

Digital technologies, including monitoring and information technologies and artificial intelligence (AI), are increasingly becoming a feature of Indigenous peoples’ lives, especially for peoples in developed and transition economies.

Yet, while data-driven technologies can drive innovation and improve human well-being, Indigenous peoples are unlikely to share equitably in these benefits given their nearly universal position of socio-economic, cultural and political marginalization.

The growing use of linked and integrated big data by governments and businesses also brings significant risks for Indigenous peoples. These include the appropriation of cultural knowledge and intellectual property; the exploitation of land and other natural resources; and the perpetuation of discrimination, stigma and ongoing marginalization.

These risks are amplified by journalistic storytelling practices that recycle well-rehearsed tropes about Indigenous dysfunction.

In this chapter we discuss some of the potential harms of digitalization and consider how Indigenous data sovereignty (ID-SOV), as an emerging site of science and activism, can mediate risks while providing pathways to benefit.

We conclude by suggesting that ID-SOV research and networks also represent valuable sources of data and data expertise that can inform more equitable, critical and just approaches to journalism involving Indigenous peoples and issues.

Indigenous Peoples and Data

There are an estimated 370 million Indigenous peoples globally, covering every continent and speaking thousands of distinct languages (United Nations, 2009). The actual global count is impossible to know as the majority of countries that encapsulate Indigenous peoples do not identify them in their national data collections (Mullane-Ronaki, 2017).

Notwithstanding these Indigenous “data deserts” and the significant global variation in Indigenous political autonomy and living standards, there is ample evidence that Indigenous people are often among the poorest population groups in their homelands, carrying the heaviest burden of disease, over-incarceration and broad spectrum inequality (Anderson et al., 2016; Stephens et al., 2006).

This shared positioning of marginalization is not coincidental; it is directly related to their history as colonized and dispossessed peoples. However, the devastating consequences of colonialism and its bedfellows, White supremacy and racism, are rarely acknowledged, let alone critiqued, in mainstream journalistic portrayals of Indigenous peoples and communities.

Indigenous peoples have always been active in what is now known as data, with ancient traditions of recording and protecting information and knowledge through, for example, art, carving, totem poles, song, chants, dance and prayers. Deliberate efforts to expunge these practices and knowledge systems were part and parcel of colonizing processes.

At the same time Indigenous peoples were made legible through the writings of European travellers, explorers and scientists who were presented as more objective, scientific and credible “knowers” of Indigenous peoples and their cultures.

Over time the racial hierarchies that justified and sustained colonialism became naturalized and embedded through ideological structures, institutional arrangements (e.g., slavery, segregation) and state classifying practices.

For example, Aboriginal and Torres Strait Islander people in Australia were specifically excluded from the national census until 1971 and this exclusion was linked to similar exclusions from basic citizenship rights such as the Age Pension (Chesterman & Galligan, 1997).

In modern times, the power to decide whether and how Indigenous peoples are counted, classified, analyzed and acted upon continues to lie with governments rather than Indigenous peoples themselves. Transforming the locus of power over Indigenous data from the nation state back to Indigenous peoples lies at the heart of ID-SOV.

Defining ID-SOV

The terminology of ID-SOV is relatively recent, with the first major publication on the topic only surfacing in 2015 (Kukutai & Taylor, 2016). ID-SOV is concerned with the rights of Indigenous peoples to own, control, access and possess data that derive from them, and which pertain to their members, knowledge systems, customs or territories (First Nations Information Governance Centre, 2016; Snipp, 2016).1

ID-SOV is supported by Indigenous peoples’ inherent rights of self-determination and governance over their peoples, country (including lands, waters and sky) and resources as described in the United Nations Declaration on the Rights of Indigenous Peoples (UNDRIP).2

Implicit in ID-SOV is the desire for data to be used in ways that support and enhance the collective well-being and self-determination of Indigenous peoples—a sentiment emphasized by Indigenous NGOs, communities and tribes (First Nations Information Governance Centre, 2016; Hudson et al., 2016).

In practice ID-SOV means that Indigenous peoples need to be the decision-makers around how data about them are used or deployed. ID-SOV thus begets questions such as:

-Who owns the data?

-Who has the power to make decisions about how data is accessed and under what circumstances?

-Who are the intended beneficiaries of the data and its application?

ID-SOV is also concerned with thorny questions about how to balance individuals’ rights (including privacy rights), risks and benefits with those of the groups of which they are a part.

The focus on collective rights and interests is an important one because it transcends the narrow focus on personal data protection and control that permeates policy and regulatory approaches such as the European Union’s General Data Protection Regulation (GDPR).

Anglo-European legal concepts of individual privacy and ownership translate poorly in Indigenous contexts where individuals are part of a broader group defined, for example, by shared genealogies or genes. In such contexts the sharing of data that encodes information about other group members cannot rest solely on personal consent but must also take account of collective rights and interests (Hudson et al., 2020).

Closely linked to ID-SOV is the concept of Indigenous data governance, which can be broadly defined as the principles, structures, accountability mechanisms legal instruments and policies through which Indigenous peoples exercise control over Indigenous data (Te Mana Raraunga, 2018a).

Indigenous data governance, at its essence, is a way of operationalizing ID-SOV (Carroll et al., 2017). It is through Indigenous data governance that Indigenous rights and interests in relation to data can be asserted (Walter, 2018).

Statistical Surveillance and Indigenous Peoples

The profiling of Indigenous populations and the targeting of services is not new; surveillance by the state, its institutions and agents have long been enduring characteristics of colonialism (Berda, 2013). Even through the official exclusion of Aboriginal and Torres Strait Islander peoples from the national census in Australia, surveillance of Aboriginal populations was a constant process (Briscoe, 2003).

What is new in the social policy arena are the opaque, complex and increasingly automated processes that shape targeting and profiling (Henman, 2018). As “data subjects” (Van Alsenoy et al., 2009), Indigenous peoples are included in a diverse range of data aggregations, from self-identified political and social groupings (e.g., tribes, ethnic/racial groups), to clusters of interest defined by data analysts on the basis of characteristics, behaviour and/or circumstances.

The position of Indigenous peoples within these data processes is not benign. Rather, while the sources of data about Indigenous peoples are rapidly evolving, the characteristics of those data as a relentless descriptive count of the various dire socio-economic and health inequalities borne by Indigenous peoples remains the same.

Walter (2016) has termed these data 5D data: Data that focus on Difference, Disparity, Disadvantage, Dysfunction and Deprivation.

Evidence to support this claim is easily found through a Google search of the term “Indigenous statistics” or by inserting the name of an Indigenous people into the search (i.e., Native American, Aboriginal and Torres Strait Islander, Maori, Native Hawaiian, First Nations, Alaskan Native). What comes up, invariably, is a sad list detailing Indigenous over-representation in negative health, education, poverty and incarceration rate data.

The impact of 5D data on Indigenous lives is also not benign. As the primary way that Indigenous peoples are positioned in the national narrative, such data shape the way the dominant non-Indigenous population understand Indigenous peoples.

These data stories that influence these narratives are frequently promulgated through media reporting. For example, Stoneham (2014) reports on a study of all articles relating to Aboriginal Health from four prominent Australian online and print media sources. Three quarters of these articles were negative, focusing on topics such as alcohol, child abuse, drugs, violence, suicide and crime, compared to just 15% of articles deemed positive (11% were rated as neutral); a ratio of seven negative articles to one positive.

Such narratives are also mostly decontextualized from their social and cultural context and simplistically analyzed, with the Indigenous population systematically compared to the (unstated) non-Indigenous norm (Walter & Andersen, 2013). The result is that in the national imagination Indigenous peoples are pejoratively portrayed as the problem rather than as peoples bearing an inordinate burden of historic and contemporary inequality.

There is growing evidence that the racial biases embedded in big data, and the algorithms developed to analyze them, will amplify, rather than reduce the impact of 5D data on Indigenous peoples (Henman, 2018).

So, while in highly developed settler states such as Aotearoa NZ and Australia the prejudicial outcomes of discriminatory policies have been unwound, to some extent, by Indigenous activism and social justice movements over many years, these emerging data practices may unintentionally entrench existing inequalities and reactivate older patterns.

With the detection (and amelioration) of social problems now increasingly deferred to algorithms, the likelihood of injustice reworking its way back into the system in ways that disadvantage Indigenous peoples rises exponentially. Reworking the old adage around data: If the algorithm data “rules” target problems where Indigenous peoples are over-represented; then the problematic Indigene will be the target.

ID-SOV in Practice

ID-SOV movements are active in the so-called CANZUS (Canada, Australia, New Zealand and the United States) states and have growing influence. The ID-SOV pioneers are First Nations in Canada.

Tired of non-Indigenous data users assuming the mantle of unbiased “experts” on First Nations peoples, community activists developed a new model which provided for First Nations collective control over their own data. The trademarked OCAP® principles assert their right to retain collective ownership of , control over, access to and possession of First Nations data and, 20 years on, have become the de facto standard for how to conduct research with First Nations (First Nations Information Governance Centre, 2016).

In Aotearoa NZ the Māori data sovereignty network Te Mana Raraunga (TMR) was established in 2015, drawing together more than a hundred Māori researchers, practitioners and entrepreneurs across the research, IT, community and NGO sectors.3

TMR has been very active in promoting the need for Māori data sovereignty and data governance across the public sector, and in 2018 took the national statistics agency to task over its handling of the New Zealand Census (Te Mana Raraunga, 2018b) which was widely reported by mainstream and Indigenous media.

TMR has also raised concerns relating to “social licence” for data use in the context of Māori data (Te Mana Raraunga, 2017) and developed its own set of Māori data sovereignty principles to guide the ethical use of Māori data (Te Mana Raraunga, 2018a).

For advocates of Māori data sovereignty, including TMR, the goal is not only to protect Māori individuals and communities from future harm and stigma, but also to safeguard Māori knowledge and intellectual property rights, and to ensure that public data investments create benefits and value in a fair and equitable manner that Māori can fully share in.

In Australia, the Maiam nayri Wingara Indigenous Data Sovereignty Collective was formed in 2016, and in 2018, in partnership with the Australian Institute of Indigenous Governance, issued a communique from a national meeting of Aboriginal and Torres Strait Islander leaders.

The communique stated the demand for Indigenous decision and control of the data ecosystem, including creation, development, stewardship, analysis, dissemination and infrastructure (Maiam nayri Wingara Indigenous Data Sovereignty Collective & Australian Indigenous Governance Institute, 2018).

Maiam nayri Wingara, alongside other Indigenous bodies, is actively advocating for changes in the way Indigenous data in Australia is conceptualized, purposed, deployed, constructed, analyzed and interpreted.

The aspiration is to activate the contribution data can make to Aboriginal and Torres Strait Islander well- being.

What is required for this to happen is a reinvention of the relationship between Indigenous data holders/generators and the Indigenous peoples to whom those data relate to one built around Indigenous data governance.

Towards a Greater Role for ID-SOV Initiatives in Data Journalism

Data journalism is well positioned to challenge rather than reinscribe the five Ds of Indigenous data. Data journalists have ample opportunities to rethink how they use data to represent Indigenous peoples and stories, and to expose the complex ways in which Indigenous data is produced, controlled, disseminated and “put to work” by government and industry.

In so doing data journalists ought not to rely on non-Indigenous data producers and users; the rise of ID-SOV networks means there are a growing number of Indigenous data experts to call on.

Many of those involved in ID-SOV work have close ties to their communities and are driven by a strong commitment to data justice and to finding ways for “good data” to empower “good outcomes.” The questions raised by ID-SOV, particularly around data ownership, control, harm and collective benefit, have wider application beyond Indigenous communities.

By engaging with ID-SOV approaches and principles, data journalists can open up meaningful spaces for Indigenous perspectives and concerns to frame their narratives, while also sharpening their lenses to hold those in power to account.

Footnotes

1. In Aotearoa New Zealand, the ID-SOV network Te Mana Raraunga defines Māori data as “digital or digitisable information of knowledge that is about or from Maori people, our language, cultures, resources or environments” (Te Mana Raraunga, 2018a).

2.www.un.org/esa/socdev/unpfii/documents/DRIPS_en.pdf

3. www.temanararaunga.maori.nz/tutohinga

Works Cited

Anderson, I., et al. (2016). Indigenous and tribal peoples’ health (The Lancet–Lowitja Institute Global Collaboration): A population study. The Lancet, 388(10040), 131–157. doi.org/10.1016/S0140-6736(16)00345-7

Berda, Y. (2013). Managing dangerous populations: Colonial legacies of security and surveillance. Sociological Forum, 28(3), 627–630.doi.org/10.1111/socf.12042

Briscoe, G. (2003). Counting, health and identity: A history of aboriginal health and demography in Western Australia and Queensland 1900–1940. Aboriginal Studies Press.

Carroll, R. S., Rodriguez-Lonebear, D., & Martinez, A. (2017). Policy brief (Ver- sion 2): Data governance for native nation rebuilding. Native Nations Institute. usindigenousdata.arizona.edu

Chesterman, J., & Galligan, B. (1997). Citizens without rights: Aborigines and Austral- ian citizenship. Cambridge University Press.

First Nations Information Governance Centre. (2016). Pathways to First Nations’ data and information sovereignty. In T. Kukutai & J. Taylor (Eds.), Indigenous data sovereignty: Toward an agenda (pp. 139–155). Australian National University Press. doi.org/10.22459/CAEPR38.11.2016.08

Henman, P. (2018). Of algorithms, apps and advice: Digital social policy and service delivery. Journal of Asian Public Policy, 12(2), 1–19. doi.org/10.1080/17516234.2018.1495885

Hudson, M., et al. (2020). Rights, interests and expectations: Indigenous perspectives on unrestricted access to genomic data. Nature Reviews Genetics, 21(6), 377–384. doi.org/10.1038/s41576-020-0228-x

Hudson, M., Farrar, D., & McLean, L. (2016). Tribal data sovereignty: Whakatōhea rights and interests. In T. Kukutai & J. Taylor (Eds.), Indigenous data sovereignty: Toward an agenda (pp. 157–178). Australian National University Press. doi.org/10.22459/CAEPR38.11.2016.09

Kukutai, T., & Taylor, J. (Eds.). (2016). Indigenous data sovereignty: Toward an agenda. Australian National University Press.

Maiam nayri Wingara Indigenous Data Sovereignty Collective & Australian Indigenous Governance Institute. (2018). Indigenous data sovereignty summit communique. web.archive.org/web/20190305225218/http://www.aigi.com.%20au/wp-content/uploads/2018/07/Communique-Indigenous-Data-Sovereignty-%20Summit.pdf

Mullane-Ronaki, M.-T. T. K. K. (2017). Indigenising the national census? A global study of the enumeration of indigenous peoples, 1985-2014 [Thesis]. University of Waikato.

Snipp, M. (2016). What does data sovereignty imply: What does it look like? In T. Kukutai & J. Taylor (Eds.), Indigenous data sovereignty: Toward an agenda(pp. 39–56). Australian National University Press.

Stephens, C., Porter, J., Nettleton, C., & Willis, R. (2006). Disappearing, displaced, and undervalued: A call to action for Indigenous health worldwide. The Lancet, 367(9527), 2019–2028. doi.org/10.1016/S0140-6736(06)68892-2

Stoneham, M. (2014, April 1). Bad news: Negative Indigenous health cover- age reinforces stigma. The Conversation.theconversation.com/bad-news-negative-indigenous-health-coverage-reinforces-stigma-24851

Te Mana Raraunga. (2017). Statement on social licence. www.temanararaunga.maori.nz/panui/

Te Mana Raraunga. (2018a). Principles of Māori data sovereignty.www.temanararaunga.maori.nz/new-page-2

Te Mana Raraunga. (2018b). Te Mana Raraunga statement on 2018 New Zealand Census of Population and Dwellings: A call for action on Māori census data. www.temanararaunga.maori.nz/panui/

United Nations. (2009). State of the world’s indigenous peoples. www.un.org/ esa/socdev/unpf ii/documents/SOWIP/en/SOWIP_web.pdf

Van Alsenoy, B., Ballet, J., Kuczerawy, A., & Dumortier, J. (2009). Social networks and web 2.0: Are users also bound by data protection regulations? Identity in the Information Society, 2(1), 65–79. doi.org/10.1007/s12394-009-0017-3

Walter, M. (2016). Data politics and Indigenous representation in Australian statistics. In T. Kukutai & J. Taylor (Eds.), Indigenous data sovereignty: Toward an agenda (pp. 79–98). Australian National University Press.

Walter, M. (2018). The voice of indigenous data: Beyond the markers of disadvantage. Griffith Review, 60, 256.

Walter, M., & Andersen, C. (2013). Indigenous statistics: A quantitative research methodology. Left Coast Press.

The news applications team at the Chicago Tribune is a band of happy hackers embedded in the newsroom. We work closely with editors and reporters to help: (1) research and report stories, (2) illustrate stories online and (3) build evergreen web resources for the fine people of Chicagoland.

It’s important that we sit in the newsroom. We usually find work via face-to-face conversations with reporters. They know that we’re happy to help write a screen scraper for a crummy government website, tear up a stack of PDFs, or otherwise turn non-data into something you can analyze. It’s sort of our team’s loss leader — this way we find out about potential data projects at their outset.

Unlike many teams in this field, our team was founded by technologists for whom journalism was a career change. Some of us acquired a masters degree in journalism after several years coding for business purposes, and others were borrowed from the open government community.

We work in an agile fashion. To make sure we’re always in sync, every morning begins with a 5-minute stand up meeting. We frequently program in pairs — two developers at one keyboard are often more productive than two developers at two keyboards. Most projects don’t take more than a week to produce, but on longer projects we work in week-long iterations, and show our work to stakeholders — reporters and editors usually — every week. “Fail fast” is the mantra. If you’re doing it wrong, you need to know as soon as possible, especially when you’re coding on a deadline!

There’s a tremendous upside to hacking iteratively, on a deadline: We’re always updating our toolkit. Every week we crank out an app or two, then, unlike normal software shops, we can put it to the back of our mind and move on to the next project. It’s a joy we share with the reporters, every week we learn something new.

All app ideas come from the reporters and editors in the newsroom. This, I believe, sets us apart from apps teams in other newsrooms, who frequently spawn their own ideas. We’ve built strong personal and professional relationships in the newsroom, and folks know that when they have data, they come to us.

Much of our work in the newsroom is reporter support. We help reporters dig through data, turn PDFs back into spreadsheets, screen-scrape websites, etc. It’s a service that we like to provide because it gets us in early on the data work that’s happening in the newsroom. Some of that work becomes a news application — a map, table, or sometimes a larger-scale website.

Before, we linked to the app from the written story, which didn’t result in much traffic. These days apps run near the top of our website, and the app links through to the story, which works nicely for both the app and the story. There is a section of the website for our work, but it’s not well-trafficked. But that’s not surprising. “Hey, today I want some data” isn’t a very big use case.

We love page views, and we love the accolades of our peers, but that’s weak sauce. The motivation should always be impact — on people’s lives, on the law, on holding politicians to account, and so on. The written piece will speak to the trend and humanise it with a few anecdotes. But what’s the reader to do when they’ve finished the story? Is their family safe? Are their children being educated properly? Our work sings when it helps a reader find his or her own story in the data. Examples of impactful, personalised work that we’ve done include our Nursing Home Safety Reports and School Report Card apps.

Written by Philippa Law and Caroline Bannock

Storm force winds and rain brought flooding and power outages to the south of the U.K. in October 2013. This event affected a lot of people, so to widen and enrich the Guardian’s coverage, we asked our readers to share their photos, videos and stories of the disruption via our user-generated content platform, GuardianWitness.

Among the contributions we received was a bizarre photo of what appeared to be a giant multicolored beach ball, at least twice the height of a double decker bus, on the loose at Old Street roundabout in London. This was one of those images that immediately evokes the question, “Is this too good to be true?” We were very aware that it could be a hoax.

We started verifying the user’s photo by running it through Google reverse image search and TinEye to verify that the image hadn’t been borrowed from another website. Users often try to show us a news event by sending pictures that have been published on other news sites, or shared on Twitter and Facebook. So a reverse image search is always the first check we make.

In the case of the rampant inflatable, Google returned no hits - which suggested the photo was either original or very recent and hadn’t been picked up by any other news organizations - yet. Good content gets published very fast!

The most important verification tool we have is a direct conversation with the user. Every contributor to GuardianWitness has to share an email address, though there’s no guarantee it’s a correct one. So we emailed the user in question to try to make contact. In the meantime we continued with our verification checks.

Usually we would verify where a photo had been taken by comparing it with images on Google Street View, but as our team is familiar with the Old Street area, we recognized the view in the photo and felt reasonably confident the picture had been taken there. Although we knew the area, we didn’t recall seeing a giant beach ball - so we searched online for earlier evidence. We found it had previously been tethered to the top of a building nearby. This finding meant the image was looking less like a hoax than it had first appeared.

We checked Twitter for mentions of the beach ball that morning and were able to confirm that there had been other sightings around the time the user claimed to have taken the photo. Our Twitter search also revealed a later photo, taken by another user, after the ball had deflated.

Finally, the user got in contact with us and, by speaking to him on the phone, we were able to confirm that he had taken the photo himself.

Having taken all these steps to verify the image, we were happy that the story held up to scrutiny. The compelling image of a runaway beach ball in the driving rain was published on the Guardian’s live-blog and was shared widely on social media.

and Gemma Bagayaua-Mendoza

A professional journalist for roughly 20 years, Gemma Bagayaua-Mendoza is the head of research and strategy at Rappler. She leads the fact-check unit as well as Rappler's research into online disinformation and misinformation.

Vernise Tantuco is a member of Rappler's research team, where she works on fact checks and studies disinformation networks in the Philippines.

In the fall of 2016, John Victorino, an investment analyst, sent Rappler a list of what he said was 26 suspicious Facebook accounts from the Philippines. We began investigating and monitoring the accounts, and quickly found the details listed in their profiles were false. Over the course of weeks of investigation, these 26 accounts led us to uncover a much more extensive network of pages, groups and accounts.

These accounts, along with a set of pages and groups they were connected to, were eventually removed by Facebook. They also inspired Rappler to create Sharktank, a tool for monitoring how information flows on Facebook. That work formed the basis of a series of investigative stories about how propaganda and information operations on Facebook affect democracy in the Philippines. The series included an investigation into the activities of the 26 fake accounts, and kicked off our continued coverage of how Facebook has been weaponized in the Philippines to spread political disinformation, harass people and undermine democracy in the country.

This case study examines how we investigated the original 26 accounts and used them to uncover much larger networks.

Verifying identities, exposing sockpuppets

Our first step in investigating the set of accounts was to try to verify if they were connected to real people. This part required good old fashioned fact-checking and began with our creating spreadsheets to track details related to the accounts, including the personal details they listed, the pages they liked and other information.

For example, Facebook user Mutya Bautista described herself as a “software analyst” at ABS-CBN, the Philippines’ largest television network. Rappler checked with ABS-CBN, who confirmed that she did not work for them.

Using reverse image search tools, we found that many of 26 accounts used profile photos of celebrities or personalities.

Bautista, for example, used a picture of Im Yoona of the Korean pop group Girl’s Generation. The Lily Lopez account, shown below, used the image of Korean actress Kim Sa-rang.

Another account, Luvimin Cancio, used an image from softcorecams.com, a porn site, as its profile photo. We identified this website as the source of the photo through the reverse image search tool TinEye.

The accounts also used similar cover photos on their profiles. Below, the cover photo of the account of Jasmin De La Torre is the same as that of Lily Lopez.

We also noticed one curious thing about the 26 accounts: These users had more groups than friends.

This was unusual because, in the Philippines, most people have friends and family abroad. Facebook basically serves as the communication channel through which people keep in touch with family and friends. So they tend to have many friends as opposed to being members of a huge number of groups.

Bautista’s friends list, which was public at the time, showed she had only 17 friends. In fact, each of the 26 accounts that we identified each had fewer than 50 friends when we discovered them in 2016.

Bautista however, was a member of over a hundred groups, including groups campaigning for then-vice presidential candidate Ferdinand Marcos Jr., a number of communities of Filipinos overseas, as well as buy and sell groups, each with members ranging from tens of thousands to hundreds of thousands. Altogether, these groups have over 2.3 million members on Facebook. Below is a list of some of the biggest groups, including their follower counts. Also included is a list of the posts Bautista made to these groups.

By combining all of these observations and associated data, we concluded that the accounts were sockpuppets: fictional identities created to bolster a particular point of view.

Pro-Marcos network

We could see from the dates associated with the first profile photos and early posts of these 26 accounts that they appeared to have been created in the last quarter of 2015, leading up to the May 2016 elections. We also found that they consistently promoted content that denied the widely documented martial law abuses that took place in the 1970s under the Marcos regime. The accounts also attacked the rivals of the former dictator’s son, vice presidential candidate Ferdinand “Bongbong” Marcos Jr.

In the example below, user Mutya Bautista shared a now-debunked claim that Bongbong’s rival — then-newly proclaimed vice president Leni Robredo — was previously married to an activist before she married her second husband, the late Interior and local government secretary Jesse Robredo. Bautista posted the story headlined “Leni Robredo was married to an anti-Marcos teen before she met Jesse?” to the group “Pro Bongbong Marcos International Power,” with the comment: “Kaya ganun na lamang ang pamemersonal kay [Bongbong Marcos], may root cause pala.” (“That’s why it’s personal against [Bongbong Marcos], there’s a root cause.”)

Another suspicious account with the name Raden Alfaro Payas shared the same article to the group “Bongbong Marcos loyalist Facebook warriors” with the exact same caption — word for word, down to the last punctuation mark — on the same day.

Fake accounts are often used to spam groups with links, and you can sometimes catch them reusing the same text when they do it. At the time, it was possible to use Facebook Graph search to look at the public posts of users in groups. However, Facebook closed off many Graph search features in 2019, including this function. As a result, it’s now necessary to go into groups and search to see what specific users have been sharing.

Connected websites

By analyzing what content the accounts shared, we were able to see that the 26 sockpuppets were promoting the same websites: Okay Dito (OKD2.com), Ask Philippines (askphilippines.com) and why0why.com, among others.

OKD2.com has published a number of hoaxes and other propaganda material favoring the Marcos family and President Rodrigo Duterte. It now masquerades as a classified ads site. But in September 2016 we found that content from the site was shared 11,900 times on Facebook, thanks in part to the sockpuppets.

Through these websites, Rappler eventually traced the potential puppet master of the 26 accounts: someone named Raden Alfaro Payas.

Tracking the puppeteers

Like many sites that Rappler monitors, OKD2.com’s current domain registration records are private. The site also does not disclose its authors or owners, and has no contact information other than a web form.

Fortunately, we were able to use historical domain records to identify a person associated with the site. Using domaintools.com, we could see that as of July 2015, OKD2.com was registered in the name of one Raden Payas, a resident of Tanauan City, Batangas. We also found that OKD2.com shared the same Google AdSense ID as other websites, such as askphilippines.com and why0why.com, that the 26 accounts were sharing. We identified the AdSense IDs on these sites by viewing the source code of pages on them and looking for a series of numbers that began with the letters “ca-pub-.” Each Google AdSense account is given a unique ID that begins with “ca-pub-,” and each page of a site that is linked to an account will have this code on it.

Along with the domain record, we also saw that one of the 26 accounts was called Raden Alfaro Payas (Unofficial). We also found another account in his name with the username “realradenpayas,” which interacted with some of the sockpuppets.

For example, he commented on a post from Luvimin Cancio that linked to a story denying the martial law atrocities under Marcos. The “real” Payas account said he was in high school during the martial law years and he “never heard” of anybody being killed or tortured.

Jump-starting the Sharktank

These 26 fake accounts and their reach inspired Rappler to create its Sharktank database and automate data collection from public Facebook groups and pages. As of August 2019, Rappler has tracked roughly 40,000 pages with millions of followers.

What began as an investigation into a set of suspicious accounts turned into a continuing study of a network of thousands of fake and real accounts, groups and pages that spread disinformation and propaganda, distorting and politics and weakening the democracy of a nation.

When we launched the Datablog, we had no idea who would be interested in raw data, statistics and visualizations. As someone pretty senior in my office said: “why would anyone want that?”.

The Guardian Datablog — which I edit — was to be a small blog offering the full datasets behind our news stories. Now it consists of a front page (guardian.co.uk/data); searches of world government and global development data; data visualizations by from around the web and Guardian graphic artists, and tools for exploring public spending data. Every day, we use Google spreadsheets to share the full data behind our work; we visualize and analyze that data, then use it to provides stories for the newspaper and the site.

As a news editor and journalist working with graphics, it was a logical extension of work I was already doing, accumulating new datasets and wrangling with them to try to make sense of the news stories of the day.

The question I was asked has been answered for us. It has been an incredible few years for public data. Obama opened up the US government’s data faults as his first legislative act, followed by government data sites around the world — Australia, New Zealand, the British government’s Data.gov.uk

We’ve had the MPs expenses scandal — Britain’s most unexpected piece of data journalism — the resulting fallout has meant Westminster is now committed to releasing huge amounts of data every year.

We had a general election where each of the main political parties was committed to data transparency, opening our own data vaults to the world. We’ve had newspapers devoting valuable column inches to the release of the Treasury’s COINS database.

At the same time, as the web pumps out more and more data, readers from around the world are more interested in the raw facts behind the news than ever before. When we launched the Datablog, we thought the audiences would be developers building applications. In fact, it’s people wanting to know more about carbon emissions or Eastern European immigration or the breakdown of deaths in Afghanistan — or even the number of times the Beatles used the word “love” in their songs (613).

Gradually, the Datablog’s work has reflected and added to the stories we faced. We crowdsourced 458,000 documents relating to MPs' expenses and we analyzed the detailed data of which MPs had claimed what. We helped our users explore detailed Treasury spending databases and published the data behind the news.

But the game-changer for data journalism happened in spring 2010, beginning with one spreadsheet: 92,201 rows of data, each one containing a detailed breakdown of a military event in Afghanistan. This was the WikiLeaks war logs. Part one, that is. There were to be two more episodes to follow: Iraq and the cables. The official term for the first two parts was SIGACTS: the US military Significant Actions Database.

News organizations are all about geography — and proximity to the news desk. If you’re close, it’s easy to suggest stories and become part of the process; conversely out of sight is literally out of mind. Before Wikileaks, we were sat on a different floor, with graphics. Since Wikileaks, we have sat on the same floor, next to the newsdesk. It means that it’s easier for us to suggest ideas to the desk, and for reporters across the newsroom to think of us to help with stories.

It’s not that long ago journalists were the gatekeepers to official data. We would write stories about the numbers and release them to a grateful public, who were not interested in the raw statistics. The idea of us allowing our raw information into our newspapers was anathema.

Now that dynamic has changed beyond recognition. Our role is becoming interpreters, helping people understand the data — and even just publishing it because it’s interesting in itself.

But numbers without analysis are just numbers, which is where we fit in. When Britain’s prime minister claims the riots in August 2011 were not about poverty, we were able to map the addresses of the rioters with poverty indicators to show the truth behind the claim.

Behind all our data journalism stories is a process. It’s changing all the time as use new tools and techniques. Some people say the answer is to become a sort of super hacker, write code and immerse yourself in SQL. You can decide to take that approach. But a lot of the work we do is just in Excel.

Firstly, we locate the data or receive it from a variety of sources, from breaking news stories, government data, journalists' research and so on. We then start looking at what we can do with the data — do we need to mash it up with another dataset? How can we show changes over time? Those spreadsheets often have to be seriously tidied up — all those extraneous columns and weirdly merged cells really don’t help. And that’s assuming it’s not a PDF, the worst format for data known to humankind.

Often official data comes with the official codes added in; each school, hospital, constituency and local authority has a unique identifier code.

Countries have them too (the UK’s code is GB, for instance). They’re useful because you may want to start mashing datasets together and it’s amazing how many different spellings and word arrangements can get in the way of that. There’s Burma and Myanmar, for instance, or Fayette County in the US — there are 11 in states from Georgia to West Virginia. Codes allow us to compare like with like.

At the end of that process is the output; will it be a story or a graphic or a visualization, and what tools will we use? Our top tools are the free ones that we can produce something quickly with. The more sophisticated graphics are produced by our dev team.

Which means we commonly use Google charts for small line graphs and pies, or Google Fusion Tables to create maps quickly and easily.

It may seem new, but really it’s not.

In the very first issue of the Manchester Guardian, Saturday 5 May, 1821, the news was on the back page, like all papers of the day. First item on the front page was an ad for a missing Labrador.

And, amid the stories and poetry excerpts, a third of that back page is taken up with, well, facts. A comprehensive table of the costs of schools in the area never before “laid before the public”, writes “NH”.

NH wanted his data published because otherwise the facts would be left to untrained clergymen to report. His motivation was that: “Such information as it contains is valuable; because, without knowing the extent to which education … prevails, the best opinions which can be formed of the condition and future progress of society must be necessarily incorrect.” In other words, if the people don’t know what’s going on, how can society get any better?

I can’t think of a better rationale now for what we’re trying to do. Now what once was a back page story can now make front page news.

Written by Tom Phillips

When Hurricane Sandy hit New York and New Jersey, I was running a blog called “Is Twitter Wrong?” an experiment at fact-checking viral images.

When a major natural disaster hits an area densely populated with heavy social media users - and media companies - one result is a huge number of images to sift through. Telling the good from the bad suddenly shot up the editorial agenda.

One particularly viral pair of images showed a shark supposedly swimming up a flooded New Jersey street. I teamed up with Alexis Madrigal from The Atlantic to try to verify these images.

One aspect of the images, shown below, is that they were strange enough to make you suspicious, yet they weren’t implausible enough to dismiss out of hand. In the end, and they proved very hard to definitively debunk.

Pre-existing images that have been misattributed (perhaps the most common form of “fake”) can often be debunked in a few seconds through a reverse image search. And pictures of major events can often be at least partly verified by finding mutually confirmatory images from multiple sources.

But neither of those work for a one-off chance sighting that’s either an original picture or an original hoax. (My experience is that verification of images that can’t be debunked/verified within a few minutes tends to take a lot longer.)

In the end, sometimes there’s no substitute for the time-consuming brute force approach of image verification: tracing an image’s spread back through social media to uncover the original; walking the streets of Google Street View to pinpoint a rough location; and/or scrolling through pages of Google Image results for a particular keyword, looking for possible source images.

In this case, the Google Image search approach paid off - we were able to find the exact image of a shark’s fin that had been Photoshopped into one of the pictures.

But even then, we were unable to say that the other image was definitively fake. It used a different shark.

Our attempts to find the origin of both shark images kept hitting the barrier of people saying, vaguely, that it was “from Facebook.” We eventually found the originating Facebook poster via a tweet directing us to a news site that credited the source. (Both the news report and Facebook posts have since vanished from the Web.) But even that didn’t entirely help, as the page owner’s other photos showed genuine flooding in the same Brigantine, New Jersey, location. He also insisted in replies to friends that the shark pictures were real. (In retrospect, he seemed to be intent mostly on pranking his social circle, rather than hoaxing the entire Internet.)

The fact that he was claiming one undoubted fake as real was enough for us to move the other shark image into the “almost certainly fake” category. But we still didn’t know for sure. It wasn’t until the next day, when the fact-checking site Snopes managed to identify the source image, that we were able to make that call with 100 percent certainty. This was the shark image that was used to create the fake:

That may be the main lesson from Sandy: Especially in rapidly developing situations, verifica- tion is often less about absolute certainty, and more about judging the level of acceptable plausibility. Be open about your uncertainties, show your work, and make it clear to the reader your estimate of error when you make a call on an image.

Written by Donie O'Sullivan

Donie O’Sullivan is a CNN reporter covering the intersection of technology and politics. He is part of the CNN Business team and works closely with CNN's investigative unit tracking and identifying online disinformation campaigns targeting the American electorate.

In the summer and fall of 2017, as the world began learning the details of Russia’s expansive effort to influence American voters through social media, it became clear that African Americans and the Black Lives Matter movement were among the main targets of the Kremlin’s campaign to sow division.

My colleagues at CNN and I spent months reporting how Russia had been behind some of the biggest Black Lives Matter (BLM) accounts on social media. As I spoke to BLM activists, I would sometimes be asked, “Do you know who runs the biggest Black Lives Matter page on Facebook?”

Incredibly, no one — including the most prominent BLM activists in the country and organizers on the ground — knew the answer. Some had understandably suspected the page might be run from Russia. But our investigation found it wasn’t Russian, or American — it was run by a white man in Australia.

The page, simply titled “Black Lives Matter,” looked legitimate. As of April 2018 it had almost 700,000 followers. It consistently shared links to stories about police brutality and inequality; it ran online fundraisers; it even had an online store that sold BLM merchandise.

It’s not unusual for a page that size to be run anonymously. Some activists don’t want to put their names on a page and risk attracting attention from trolls or scrutiny from law enforcement looking to shut down protests. Outside the U.S., the ability for activists to run pages anonymously has been critical to digital activism and key to some movements. (It was also precisely what Russia exploited, adding to suspicions that this BLM was connected.)

Around the time I began paying attention to this mysterious page, Jeremy Massler, a freelance investigator and incredible online sleuth, reached out with a tip. Massler had looked at the domain registration records of websites that the huge BLM Facebook page was consistently linking to. Although the domains had been registered privately, he found one of them had, for a period in 2016, belonged to a person in Perth, Australia, named Ian MacKay — a white man.

Massler contacted MacKay, who told him he bought and sold domains as a hobby and had nothing to do with the Facebook page. It was the same excuse MacKay, a middle-aged union official, gave me when I reached him by phone a few months later. But by that time we’d found that MacKay had registered dozens of website names, many relating to black activism.

Despite my concerns about the page and the fact that several activists told me they were suspicious of it, I didn’t find MacKay’s explanation unbelievable on its face. Domain names can be valuable, and people buy and sell them all the time. The fact he had also registered and sold domains that were not related to black activism made his case even more credible. But then something strange happened. A few minutes after I spoke to MacKay, the Facebook page came down. It hadn’t been taken down by Facebook, but by whoever was running it — and it hadn’t been deleted, only temporarily removed.

That seemed suspicious, so Massler and I began to dig more.

The Facebook page, which came back online in the weeks after my call with MacKay, had during its lifetime promoted fundraising campaigns ostensibly for BLM causes.

In one instance, it claimed to be raising money for activists in Memphis, Tennessee. But when I spoke to activists there, no one knew anything about the fundraiser or where the money might have gone. Other activists even told us that, suspecting it was a scam, they had reported the page to Facebook. But the company hadn’t taken any action.

As I started to contact the multiple online payment and fundraising platforms the page had used, those companies began removing the fundraisers, saying they had broken their rules. Citing user privacy, none of the payment companies provided me with information on the record about where the money was going. This is a common challenge. Citing their privacy policies, platforms and digital services rarely reveal the names or contact information of account holders to the press.

I later learned from a source familiar with some of the payments processed that at least one account was tied to an Australian bank account and IP address. Another source told me that around $100,000 had been raised. Developing sources at tech companies who are willing to tell you more information than the company will say on the record is becoming increasingly important as many stories cannot be uncovered purely using open source information as scammers and bad actors become more sophisticated.

I brought this information to Facebook to comment for the story and told them I had evidence the page was linked to Australia, that payment companies had removed the campaigns after they investigated, and that we knew some of the money was going to Australia. A Facebook spokesperson said the social media platform’s investigation “didn't show anything that violated our Community Standards.”

It wasn’t until shortly before publishing our story — and only after I raised my concern about Facebook’s investigation and its spokesperson’s response to a more senior Facebook employee — that Facebook took action and removed the page.

The Australian workers’ union where MacKay worked launched an investigation of its own after CNN’s report. By the end of the week it had fired MacKay and a second official it said was also involved in the scam.

What was particularly notable about this story was the array of techniques that Massler and I used to get it over the line. We relied heavily on archive sites like the Wayback Machine that allowed us to see the look of websites the page had been linking to and the page itself before it came on our radar. This was particularly useful, as after Massler initially contacted MacKay the people behind the page began trying to cover some of their tracks.

We also used services that track domain registrations, including DomainTools.com, to investigate the sites MacKay had registered and also to find his direct contact details. Massler also extensively used Facebook Graph Search (a tool no longer available) to track the fake Facebook profile accounts that had been set up to promote the page in Facebook Groups. Interrogation of open source information and use of online research tools, like those used to access domain records, are vital instruments — but they are not the only ones.

The simple act of picking up the phone to talk to MacKay and developing sources to provide information that would otherwise not be made public — traditional journalism techniques — were critical in exposing this scam.

Abstract

This chapter gives an insider view of the landscape of data journalism in China, its key players and data culture, as well as some practical tips.

Keywords: China, data culture, citizen participation, open data, data journalism, data visualization

A couple of years ago, I delivered a presentation introducing data journalism in China at the Google News Summit, organized by Google News Lab. It was a beautiful winter day in the heart of Silicon Valley, and the audience comprised a packed room of a hundred or so senior media professionals, mainly from Western countries.

I started by asking them to raise their hands if they think, firstly, that there is no good data in China, and secondly, that there is no real journalism in China. Both questions got quite some hands up, along with some laughter.

These are two common beliefs, if not biases, that I encounter often when I attend or speak at international journalism conferences. From my observations over the past six years, far from there being no data, in fact a vast quantity of data is generated every day in China, and of rapidly improving quality and broader societal relevance.

Instead of no “real” journalism being done, there are many journalists producing important stories every day, although not all of them are ultimately published.

Issue-driven Data Creation

Data stories were being produced even before the term “data journalism” was introduced in China. While nowadays we normally use the term “data-driven stories” in China, there was a period when we saw the contrary: Instead of data being the driver of stories, we witnessed stories, or particular issues, driving the production of data. This typically occurred in relation to issues that resonate with regular citizens, such as air pollution.

Since 2010, the Ministry of the Environment has published a real-time air pollution index, but one important figure was missing.1

The data on particulate matter (PM), or pollutants that measure less than 2.5 micrometres in diameter, which can cause irreversible harm to human bodies, was not published.

Given the severity of air pollution and the lack of official data on PM2.5, a nationwide campaign started in November 2011 called “I test the air for the motherland.” The campaign advocated for every citizen to contribute to monitoring air quality and to publish the results on social media platforms.2

The campaign was initiated by an environmental non-profit. The testing equipment was crowd-funded by citizens, and the non-profit organization provided training to interested volunteers.

This mobilization gained broader momentum after a few online influencers joined forces, including Pan Shiyi, a well-known business leader, who then had more than 7 million followers on Sina Weibo, one of China’s most widely used social media platforms (Page, 2011).

After two years of public campaigning, the data on PM2.5 was finally included in the government data release. It was a good start, but challenges remained. Doubts about the accuracy of the data were prompted by discrepancies between the data released by the government and that released by the U.S. embassy in China (Spegele, 2012).

The data was also not journalist-friendly. Despite hourly updates from more than a hundred cities, the information was only provided on a rolling basis on the web page, with no option to download a data set in any format. Although data has been centralized, historical data is not publicly accessible. In other words, without being able to write a script to scrape the data every hour and save it locally, it is impossible to do any analysis of trends over time or undertake comparisons between cities.

That is not the end of the story. Issue-driven data generation continues. When the data is not well structured and when data journalists struggle due to limited technical skills, civil society and “tech geeks” step in to provide support.

One early example back in 2011 was PM25.in, which scrapes air pollution data and releases it in a clean format. The site claims more than 1 billion search queries since they started operating.3

Another example is Qing Yue, a non-governmental organization which collects and cleans environmental data from government websites at all levels, and then releases it to the public in user-friendly formats. Their processed data is widely used not only by data teams in established media outlets but also by government agencies themselves for better policymaking.

The generation of data and the rising awareness around certain issues have gone hand in hand.

In 2015, a documentary investigating the severity of air pollution took the country by storm. The self-funded film, entitled Under the Dome, exposed the environmental crisis of noxious smog across the country and traced the roots of the problem and the various parties responsible (Jing, 2015).

The film has been compared with Al Gore’s An Inconvenient Truth in both style and impact. The storytelling featured a lot of scientific data, charts explaining yearly trends, and social network visualizations of corruption within environment and energy industries. As soon as it was released online, the film went viral and reached 200 million hits within three days, before it was censored and taken down within a week.

But it had successfully raised public awareness and ignited a national debate on the issue, including around the accessibility and quality of air pollution data. It has also successfully made the country’s leadership aware of the significance of the issue.

Two weeks after the release of the documentary, at a press conference held by the National People’s Congress, Premier Li Keqiang addressed a question about air pollution which referred to the film, admitting that the government was failing to satisfy public demands to halt pollution. He acknowledged some of the problems raised by the documentary, including lax enforcement of pollution restrictions, and emphasized that the government would impose heavier punishments to cut the toxic smog (Wong & Buckley, 2015). At the end of August 2015, the new Air Pollution Prevention and Control Law was issued, and it was implemented January 2016 (Lijian et al., 2015).

Air pollution is only one example illustrating that even when data availability or accessibility pose a challenge, public concern with issues can lead to citizen contributions to data generation, as well as to changes in government attitudes and in the availability of public sector data on the issues at hand. In more established ecosystems, data may be more readily available and easy to use, and the journalist’s job more straightforward: To find data and use it as a basis for stories.

In China the process can be less inear, and citizens, government, civil society and the media may interact at multiple stages in this process. Data, instead of just serving as the starting point for stories, can also come into the picture at a later stage to enable new kinds of relations between journalists and their publics.

Evolving Data Culture

The data environment in China has been changing rapidly in the past decade. This is partly driven by the dynamics described thus far in this chapter, and partly due to other factors, including the global open data movement, rapidly growing Internet companies and a surprisingly high mobile penetration rate. Data culture has been evolving around these trends as well.

Government legislation provides the policy backbone for data availability. To the surprise of many, China does have laws around freedom of information. The State Council Regulations on the Disclosure of Government Information was adopted in 2007 and came into force on May 1, 2008. The law has a disclosure mandate and affirms a commitment to government transparency. Following the regulation, government agencies at all levels set up dedicated web pages to disclose information they hold, including data sets.

However, although it gave journalists the right to request certain data or information from the authorities, in the first three years since the law was enforced, there are no publicly known cases of any media or journalists requesting data disclosure, according to a 2011 study published by Caixin, a media group based in Beijing and known for investigative journalism.4

The study revealed that, in 2010, the Southern Weekly, a leading newspaper, only got a 44% response rate to a request sent to 29 environmental bureaus to test their degree of compliance with the law. Media organizations do not usually have a legal team or other systems to support journalists to advance their investigations and further their information requests. In another instance, one journalist who, in his personal capacity, took the government to the court for not disclosing information, ended up losing his job. The difficulties and risks that Chinese journalists encounter when leveraging legal tools can be much greater than those experienced by their Western peers.

China is also responding to the global open data movement and increasing interest in big data. In 2012, both Shanghai and Beijing launched their own open data portals. Each of them holds hundreds of data sets on issues such as land usage, transportation, education and pollution monitoring. In the following years, more than a dozen open data portals have been set up, not only in the biggest cities, but also in local districts and less-developed provinces.

The development was rather bottom-up, without a template or standard structure for data release at the local level, which did not contribute to the broader comparability or usability of this data.

By 2015, the State Council had released the Big Data Development Action Plan, where open data was officially recognized as one of the ten key national projects, and a concrete timeline for opening government data was presented.5 However, official data is not always where journalists start, and also not always aligned with public interests and concerns.

On the other hand, the private sector, especially the technology giants such as Alibaba or Tencent, have over the years accumulated huge amounts of data.

According to its latest official results, Alibaba’s annual active consumers reached 601 million by September 30, 2018 (“Alibaba Group Announces,” 2018). The e-commerce data from such a strong user base—equivalent to the entire Southeast Asian population—can reveal lots of trading trends, demographic shifts, urban migration directions, consumer habit changes and so on. There are also vertical review sites where more specific data is available, such as Dianping, the Chinese equivalent of Yelp. Despite concerns around privacy and security, if used properly, those platforms provide rich resources for data journalists to mine.

One outstanding example in leveraging big data is the Rising Lab, a team under the Shanghai Media Group, specializing in data stories about urban life.6 The Lab was set up as an answer to the emerging trend of urbanization: China has more than 600 cities now, compared to 193 in 1978, with 56% of the population living in urban areas, according to a 2016 government report (“Gov’t Report: China’s Urbanization,” 2016). Shifting together with the rapid urbanization is the rise of Internet and mobile use, as well as lifestyle changes, such as the rapid adoption of sharing economy models. These trends are having a big impact on data aggregation.

With partnership agreements and technical support from tech companies, the Lab collected data from websites and apps frequently used by city dwellers. This data reflected various aspects of urban life, including property prices, numbers of coffee shops and bars, numbers of co-working spaces, and quality of public transportation.

Coupled with its original methodology, the Lab has produced a series of city rankings taking into account aspects such as commercial attractiveness, level of innovation and diversity of life (Figure 10.1). The rankings and the stories are updated every year based on new data, but follow the same methodology to ensure consistency. The concept and stories have been well received by the public and have begun to influence urban planning policies and companies’ business decisions, according to Shen Congle, director of the Lab (Shen, 2018).

The Lab’s success illustrates the new dynamics emerging between data providers, journalists, and citizens. It shows how softer topics have also become a playground for data journalism, alongside other pressing issues, such as the environmental crisis, corruption, judicial injustice, public health and money laundering. It also explores new potential business models for data journalism, as well as how data-based products can bring value to governments and businesses.

Readers’ news consumption practices have also had an impact on the development of data journalism. Two aspects deserve attention here, one being visual news consumption and the other, mobile news consumption. Since 2011, infographics have become popular thanks to a few major news portals’ efforts to build dedicated vertices with infographics stories, mostly driven by data. In 2014, the story of the downfall of the former security chief Zhou Yongkang, one of the nine most senior politicians in China, was the biggest news of the year.

Together with the news story, Caixin produced an interactive social network visualization (Figure 10.2) to illustrate the complex network around Zhou, including 37 people and 105 companies or projects connected to him, and the relationship between these entities, all based on the 60,000-word investigative piece produced by its reporting team. The interactive received 4 million hits within one week, and another 20 million views on social media, according to Caixin.7 The wide circulation of this project brought new kinds of data storytelling to new publics, and created an appetite for visual stories which didn’t exist before.

Almost at the same time, the media industry was welcoming the mobile era. More and more data stories, like any other online content in China, are now disseminated mostly on mobile. According to the China Internet Network Information Center (CNNIC), more than 95% of Internet users in the country used a mobile device to access the Internet in 2016 (Chung, 2017). WeChat, the domestic popular messaging app and social media platform, reached 1 billion users in March 2018 (Jao, 2018).

The dominance of mobile platforms means data stories in China are now not only mobile-first, but in many cases mobile-only. Such market demand led to a lot of lean, simple and sometimes creative interactives that are mobile friendly.

In short, the data culture in China has been evolving, driven by various factors from global movements to government legislation, from public demand to media requests, from new generations of data providers, to new generations of news consumers. The interdependent relationships between players have created very complex dynamics, where constraints and opportunities coexist. Data journalism has bloomed and advanced along its own path in China.

Practical Tips

This final section is aimed at readers of this book who are looking to work on China-related stories and wondering where to get started. It will not be easy. If you are not a Chinese language speaker, you will be faced with language barriers, as most data sources are only available in Chinese.

Next you will be faced with common issues pertaining to working with data: Data accuracy, data completeness, data inconsistency, etc., but we will assume that, as a reader of this book, you have the skills to deal with these issues, or at least a willingness to learn.

A good way to start would be to identify the biggest players in data journalism in China. Quite a few of the leading media outlets have data teams, and it is good to follow their stories and talk to their reporters for tips. Here are a few you should know: The Data Visualisation Lab (Caixin), Beautiful Data Channel (The Paper), The Rising Lab (Shanghai Media Group), and DT Finance.6

The second question pertains to where to find data. A comprehensive list of data sources would be a separate book, so here are just a few suggestions to get started. Start with government websites, both central ministries and local agencies.

You would need to know which department is the right one for the data you are looking for, and you should check both the thematic areas of ministries (for example, the Ministry of Environmental Protection) and the dedicated data website at the local level, if it exists.

There will be data that you don’t even expect—for example, would you expect that the Chinese government published millions of court judgements after 2014 in full text? Legal documents are relatively transparent in the United States but not in China. But the Supreme People’s Court (SPC) started a database called China Judgments Online doing just that.

Once you find some data that could be useful online, make sure to download a local copy. It is still common that data is not available online. Sometimes the data is published in the form of annual government reports which you can order online, or available only in paper archives. For example, certain government agencies have the records of private companies but not all of these are available online.

If the data is not released at all by the government, check if any user-generated content is available. For example, data on public health is very limited, but there are dedicated websites with information on hospital registrations or elderly centres, among others. Scraping and cleaning this data would help you gain a good overview of the topic.

It is also recommended to utilize databases in Hong Kong, anything from official ones like the Hong Kong Companies Registry, to independent ones such as Webb-site Reports. As mainland China and Hong Kong are becoming politically and financially closer, more information is available there, thanks to Hong Kong’s transparent environment and legal enforcement, which may be valuable for tracing money.

There is also data about China not necessarily held in China. There are international organizations or academic institutions that have rich China-related data sets. For example, The Paper used data from NASA and Harvard University in one of its latest stories.

Last but not least, while some challenges and experience are unique to China, a lot of them could potentially provide useful lessons for journalists in other countries, where the social, cultural and political arrangements have a different shape but similar constraints.

Footnotes

1. www.gov.cn/jrzg/2010-11/25/content_1753524.htm (Chinese language, Xinhua News Agency, National air quality real-time release system launched in Beijing, November 25, 2010)

2. www.bjep.org.cn/pages/Index/40-1699?rid=2782 .. (Chinese language)

3. chrome.google.com/webstore/detail/pm25b....

4.finance.ifeng.com/leadership/gdsp/20110901/4512444.html (Chinese language)

5. www.gov.cn/zhengce/content/2015-09/05/content_10137.htm (Chinese language)

6. zhuanlan.zhihu.com/therisinglab (Chinese language)

Works Cited

Alibaba Group announces September quarter 2018 results. (2018, November 2). Business Wire. www.businesswire.com/news/home/20181102005230/en/Alibaba-Group-Announces-September-Quarter-2018-Results

Edward Wong and Chris Buckley, ‘Chinese Premier Vows Tougher Regulation on Air Pollution’, New York Times, March 15, 2015

Chung, M.-C. (2017, February 2). More than 95% of Internet users in China use mobile devices to go online. EMarketer.www.emarketer.com/Article/ More-than-95-of-Internet-Users-China-Use-Mobile-Devices-Go-Online/1015155

Gov’t report: China’s urbanization level reached 56.1%. (2016, April 20). CNTV. english.www.gov.cn/news/video/2016/04/20/content_281475331447793.htm

Jao, N. (2018, March 5). WeChat now has over 1 billion active monthly users world-wide. TechNode. technode.com/2018/03/05/wechat-1-billion-users/

Jing, C. (2015, February 28). Chai Jing’s review: Under the dome—investigating China’s smog. www.youtube.com/watch?v=T6X2uwlQGQM

Lijian, Z., Xie, T., & Tang, J. (2015, December 30). How China’s new air law aims to curb pollution. China Dialogue. www.chinadialogue.net/article/show/ single/en/8512-How-China-s-new-air-law-aims-to-curb-pollution

Page, J. (2011, November 8). Microbloggers pressure Beijing to improve air pollution mon-itoring. The Wall Street Journal. blogs.wsj.com/chinarea... internet-puts-pressure-on-beijing-to-improve-air-pollution-monitoring/ internet-puts-pressure-on-beijing-to-improve-air-pollution-monitoring/

Shen, J. (2018, October). Data journalism in China panel. Uncovering Asia, Investiga- tive Journalism Conference, Seoul. 2018.uncoveringasia.org/schedule

Spegele, B. (2012, January 23). Comparing pollution data: Beijing vs. U.S. embassy on PM2.5. The Wall Street Journal. https://blogs.wsj.com/chinarealtime/2012/01/23/comparing-pollution-data-beijing-vs-u-s-embassy-on-pm2-5/

Wong, E., & Buckley, C. (2015, March 15). Chinese premier vows tougher regulation on air pollution. The New York Times. www.nytimes.com/2015/03/16/world/asia/chinese-premier-li-keqiang-vows-tougher-regulation-on-air-pollution.html

The PISA based Wealth Comparison project is an interactive visualization that enables comparison of standards of living in different countries. The interactive uses data from the OECD’s comprehensive world education ranking report, PISA 2009, published in December 2010. The report is based on a questionnaire which asks fifteen-year-old pupils about their living situation at home.

The idea was to analyze and visualize this data to provide a unique way of comparing standards of living in different countries.

First of all our in-house editorial team decided which facts seemed to be useful to make living standards comparable and should be visualized, including:

With the help of the internal design team these facts were translated into self-explanatory icons. A front end design was built to make comparison between the different countries like in a card-game possible.

Next we contacted people from the German Open Data Network to find developers who could help with the project. This community of highly motivated people suggested Gregor Aisch, a very talented information designer, to code the application that would make our dreams come true (without using Flash — which was very important to us!). Gregor created a very high quality and interactive visualization with a beautiful bubble-style, based on the Raphaël-Javascript Library.

The result of our collaboration was a very successful interactive which got a lot of traffic. It is easy to compare any two countries, which makes it useful as a reference tool. This means that we can re-use it in our daily editorial work. For example if we are covering something related to the living situation in Indonesia, we can quickly and easily embed a graphic comparing the living situation in Indonesia and Germany. The know-how transferred to our in house-team was a great investment for future projects.

At the Zeit Online, we’ve found that our data journalism projects have brought us a lot of traffic and have helped us to engage audiences in new ways. For example, there was a wide coverage about the situation at the nuclear plant in Fukushima after the Tsunami in Japan. After radioactive material escaped from the power plant, everyone within 30 kilometres of the plant was evacuated. People could read and see a lot about the evacuations. Zeit Online found a innovative way to explain the impact of this to our German audience. We asked: How many people live near a nuclear power plant in Germany? How many people live within a radius of 30 kilometres? A map shows how many people would have to be evacuated in a similar situation in Germany. The result: lots and lots of traffic and the project went viral over the social media sphere. Data journalism projects can be relatively easily adapted to other languages. We created an English language version about proximity to nuclear power plants in the US, which was a great traffic motor. News organizations want to be recognized as trusted and authoritative sources amongst their readers. We find that data journalism projects combined with enabling our readers to look and reuse the raw data brings us a high degree of credibility.

For two years the R&D Department and the Editor-in-Chief at the Zeit Online, Wolfgang Blau, have been advocating data journalism as an important way to tell stories. Transparency, credibility and user engagement are important parts of our philosophy. That is why data journalism is a natural part of our current and future work. Data visualizations can bring value to the reception of a story, and are an attractive way for the whole editorial team to present their content.

For example, on 9th November 2011 Deutsche Bank pledged to stop financing cluster bomb manufacturers. But according to a study by non-profit organzation Facing Finance, the bank continued to approve loans to producers of cluster munitions after that promise was made. Our visualization based on the data shows the various flows of money to our readers. The different parts of the Deutsche Bank company are arranged at the top, with the companies accused of involvement in building cluster munitions at the bottom. In between, the individual loans are represented along a timeline. Rolling over the circles shows the details of each transaction. Of course the story could have been told as a written article. But the visualization enables our readers to understand and explore the financial dependencies in a more intuitive way.

To take another example: the German Federal Statistic Office has published a great dataset on vital statistics for Germany, including modelling various demographic scenarios up until 2060. The typical way to represent this is a population pyramid — such as the one from the Federal Statistics Agency.

With our colleagues from the science department we tried to give our readers a better way to explore the projected demographic data about our future society. With our visualization, we present a statistically representative group of 40 people of different ages from the years 1950 till 2060.They are organised into eight different groups. It looks like a group photo of German society at different points in time. The same data visualized in a traditional population pyramid gives only a very abstract feeling of the situation, but a group with kids, younger people, adults and elderly people means our readers can relate to the data more easily. You can just hit the play button to start a journey through eleven decades. You can also enter your own year of birth and gender to become part of the group photo: to see your demographic journey through the decades and your own life expectancy.

Written by Henk van Ess

Henk van Ess is an assessor for Poynter’s International Fact-Checking Network. He is obsessed with finding stories in data. Van Ess trains worldwide media professionals in internet research, social media and multimedia. His clients include NBC News, BuzzFeed News, ITV, Global Witness, SRF, Axel Springer, SRF and numerous NGOs and universities. His websites whopostedwhat.com and graph.tips are heavily used to filter social media. He is @henkvaness on Twitter.

For decades, Canadian flight attendant Gaëtan Dugas was known as “Patient Zero,” the first man to bring AIDS to the United States. This distinction, which was reinforced by books, films and countless news reports made him the “arch-villain of an epidemic that would eventually kill more than 700,000 people in North America.”

But that was not the case. Bill Darrow, an investigator with the Centers for Disease Control and Prevention, interviewed Dugas and filed him as “Patient O, as in “Out-of-California.” It was soon misread as the number 0, setting off a chain reaction of misinformation that persisted until recently.

It’s also possible for a journalist to focus on the wrong patient 0 if you don’t know how to search properly. This chapter helps you to find primary sources online by getting rid of superficial results and digging deeper.

1. Risks of consulting primary sources and how to fix them

Journalists love online primary sources. Firsthand evidence can be found in a newspaper article, a scientific study, a press release, social media or any other possible “patient zero.”

Performing a basic keyword search on an official government site can make you think “what you see is what they got.” That is often not true. Here is an example. Let’s go to the U.S. Securities and Exchange Commission, a source used to find financial information about U.S. citizens as well as businesspeople from all over the world. Let’s say we want to find the first occurrence of the phrase “Dutch police” in sec.gov. The built-in search engine of the SEC can help:

You get just one hit — a document from 2016. So the SEC mentions the Dutch police only once, in 2016, right?

Wrong. The first mention on sec.gov was in 2004, 12 years earlier, in a declassified, encrypted mail:

You won’t see this in the search results from the search bar on sec.gov, even though this information does come straight from its website. Why the difference?

By default, you should distrust search engines from primary sources. They can give you a false impression of the actual content of the website and its associated databases. The proper way to search is to perform a “primary source check.”

Primary source check

Step 1: Look at the failing link

The search result from the SEC provided us with just one source:

Let’s work with that disappointment. First, get rid of “https://www,” the first part of the link. Watch out for the first backslash after that (/) — in this case it’s before the word “litigation/”

That’s the part we need: sec.gov

2. Second step: Use “site:”

Go to a generic search engine. Start with the query (“Dutch police”) and end with “site:” followed directly with the URL (no spaces). This is the formula for finding out if an original source shows you everything:

Including specific folders

You can now adapt the “primary source formula” to your needs. Let’s go to the press release section of the New Jersey Courts website. Say you want to find out when the Mercer County Bar Association sponsored a Law Day program, but you can’t find the primary source in the title of any press release. The “Mercer County Bar Association” is not visible in any title.

Now look at the URL of that page full of poorly indexed press releases:

The public relations material is filed away in the folder /public. That should be included in your Google search:

And there you are:

Predicting folders

China has a Ministry of Ecology and Environment. Do they have English documents about the German company Siemens? With the following formula, you get Chinese and English documents in the search results:

If you want to filter to see only the English ones, maybe they used the word English in the link? Try it out. It works:

2. Following the trail of documents

Sometimes the information we need isn’t contained on a webpage, but is actually in a document hosted on a website. Here’s how to follow the document trail using Google formulas.

Ross McKitrick is an associate professor in the Economics Department at the University of Guelph, Ontario. Back in 2014, he did a presentation for a climate skeptic group. Let’s try to find the invitation for that meeting. We know it was held on May 13, 2014, and was the 11th Annual Luncheon organized by the “Friends of Science (FOS).” If we search Google for these terms we come up empty:

Why? Because the word invitation is not in many invitations. It’s the same with the word interview. Many interviews don’t contain the word interview. Even most maps don’t have the word map explicitly written on it. My advice? Stop guessing and Go Zen.

Step 1: Establish the document type

Try to find the common denominator of any online invitation. It's often a PDF document. Search for just that with “filetype:pdf” and you might find it.

Step 2: Be (climate) neutral

You don’t know the exact wording of the invitation. But what you do know, is that the YouTube video was from a May 13, 2014, event. It’s feasible that the date is mentioned in the invitation. (Be sure to search for both the cardinal and ordinal forms, May 13 and May 13th.)

Step 3: Who is involved?

We know the organizer is “Friends of Science” and its website is friendsofscience.org.

When you combine all three steps, the query in Google will be:

There it is in the first hit: the invitation for the event.

The FOS, based in Calgary, is frequently labeled a climate denial group and is funded in part by the oil and gas sector So how would we craft a query to find out more information about it and its network of supporters and funders?

Step 1: Include target

“Friends of Science” results in too many hits, so include also “Calgary.”

Step 2: Include “filetype”

Go for the next best thing for any official document, “filetype:pdf.”

Step 3: Exclude your target’s website

Exclude the target’s website Friendsofscience.org by adding “-site:friendsofscience.org.” This helps you find information from outside parties.

The full query is:

Because you searched for the target in official documents, but not from its own website, you find some brothers in arms and those who are critical of the organization:

3. Filtering social media for primary sources

YouTube

YouTube’s search tool has a problem: it won’t let you filter for videos that are older than one year. If you want to find a video of a tour in Prague from Oct 11, 2014, this is the roadblock you will hit:

To solve this, manually enter the preferred date into a Google.com search by using the “Tools” menu on the far right. Then select “Any time” and “Custom Range.” Now we get the results we need:

Twitter

Despite the power of the “site:” search operator, you’ll be disappointed if you use it in Google to try searching Twitter. For example, we could try this query to find when I tweeted about the Verification Handbook for the first time:

But it returns you only one hit as of this writing. Generic search engines like Google often struggle to deliver quality results from the trillions of posts on Twitter, or on big platforms such as Facebook and Instagram. The answer for Twitter is to use its Advanced Search functionality and add keywords, username and time period, as shown here:

Don't forget to click on “Latest" on the menu at the top of the search results page so you can view the results in reverse chronological order. By default, Twitter sorts your results by what it considers to be the top tweets.

Facebook

Using “site:” on Facebook is also not ideal, but we can make its native search tool fit our needs. Let’s say for example you want to see posts from March 2019 about strawberry cake from people in Brooklyn. Follow these steps:

Step 1: Type in query

Step 2: Click on posts

Step 3: Define location

Step 4: Choose a date

And there you are:

Instagram

To search Instagram for posts from a specific date in a specific location, you can go to my site, whopostedwhat.com, and fill in your query:

Abstract

Documenting land conflicts to tell deeper, more nuanced and bigger stories of land and its relationship with India’s diverse society.

Keywords: land conflict, India, data journalism, databases, collaboration

Land is a scarce resource in India. The country only has 2.4% of the world’s land area but supports over 17% of the world’s population. As one of the world’s fastest-growing economies, it requires large swathes of lands to fuel its ambitious agenda of industrial and infrastructure growth. At least 11 million hectares of land are required for development projects in the next 15 years. But a huge section of India’s population—mostly marginalized communities—depend on land for their sustenance and livelihood. Over 200 million people depend on forests while 118.9 million depend on farming land in India.

These competing demands cause conflicts. In many cases land is forcefully acquired or fraudulently grabbed by the state or private interests, dissenters are booked by the state agencies under false charges, compensation is paid partially, communities are displaced, houses are torched, and people get killed. Social disparities around caste, class and gender also fuel land struggles. Climate change-induced calamities are making land-dependent communities further vulnerable to displacements. All this is reflected in the many battles taking place over land across India.

As journalists writing about development issues in India, we come across many such conflicts. However, we realized it was not easy to sell those stories happening in remote corners of India to the editors in New Delhi. The mainstream media did not report on land conflicts except the ones that turned fatally violent or that were fought in the national courts. Sporadic Reporting by a few journalists had little impact. Voices of the people affected by such conflicts remained unheard. Their concerns remained unaddressed.

The reason, we thought, was that the reporters and the editors looked at the conflicts as isolated incidents. We knew land conflicts were one of the most important stories about India’s political economy. But the question was how to sell it to editors and readers. We thought that if journalists could scale up their reporting on individual cases of conflict to examine broader trends, their stories could not only have wider reach but might also show the intensity of various kind of conflicts and their impact on people, the economy and the environment. The biggest challenge to achieving this was the lack of a database which journalists could explore to see what trends are emerging around specific kinds of conflicts, such as those over roads, townships, mining or wildlife-protected areas. There was no such database of ongoing land conflicts in India. So we decided to build one.

In November 2016, we started Land Conflict Watch, a research-based data journalism project which aims to map and document all ongoing land conflicts in India. We developed a documentation methodology in consultation with academics working on land governance. We put together a network of researchers and journalists, who live across the country, to document the conflicts in their regions following this methodology.

For the purpose of this project, we defined land conflict as any situation that has conflicting demands or claims over the use or ownership of land, and where communities are one of the contesting parties. Ongoing conflicts where such demands or claims have already been recorded in a written or audio-visual format at any place, from the village level to the national level, are included. These records could be news reports, village assembly resolu- tions, records of public consultation for development projects, complaints submitted by people to government authorities, police records or court documents. Conflicts such as property disputes between two private parties or between a private party and the government are excluded unless they directly affect broader publics.

The researchers and journalists track national and local media coverage about their regions and interact with local activists, community organizations and lawyers to find cases of conflict. They then collect and verify information from publicly available government documents and independent studies, and by talking to affected parties. Data such as location of conflict, reasons behind the conflict, number of affected people, affected area, land type—whether private, common or forest—names of the government and corporate agencies involved, and a narrative summary of the conflict are documented.

Researchers file all the data into reporting-and-review software built into the Land Conflict Watch website. Data is examined and verified by dedicated reviewers. The software allows to-and-fro workflow between the researchers and the reviewers before the data is published. The dashboard, on the portal, not only presents the macro picture of the ongoing conflicts at the national level but zooms in to give details of each conflict, along with the supporting documents, at the micro level. It also provides the approximate location of the conflict on an interactive map.

About 35 journalists and researchers are currently contributing. As of September 2018, the project had documented over 640 cases. These conflicts affect close to 7.3 million people and span over 2.4 million hectares of land. Investments worth USD186 billion are attached to projects and schemes affected by these conflicts.

As a conflict is documented, it is profiled on the portal as well as on social media to give a heads-up to national journalists and researchers. The project team then collaborates with journalists to create in-depth, investigative stories at the intersection of land rights, land conflicts, politics, economy, class, gender and the environment using this data. We also collaborate with national and international media to get these stories published. Many of these stories have been republished by other mainstream media outlets. We have also conducted training to support journalists in using the database to find and scale up stories around land governance.

Land Conflict Watch is an ongoing project. Apart from designing stories, we also work with academics, researchers and students to initiate public debates. Land Conflict Watch’s data has been cited by policy think tanks in their reports. Land-governance experts have written op-eds in national newspapers using the data. We regularly get requests from students at Indian and foreign universities to use our data in their research. Non-profit organizations use land conflict data, documents and cases to strengthen their campaigns to fight for the land rights of conflict affected communities. The stories inform people and help shape discourse around land rights and governance-related issues in India.

Written by Malachy Browne

The convergence of affordable smartphone and camera technology, ubiquitous Internet access and social media is largely responsible for the explosion in citizen-powered news coverage. One byproduct of this is an enormous amount of video being uploaded and shared every minute, every hour.

The revolution in information technology is not over and the volume of newsworthy user- generated content will only grow. Journalists have a new responsibility - to quickly gather, verify and ascertain the usage rights of UGC. Traditional values of investigation apply, but a new skillset is required for media such as video.

Verifying video from an unknown source on social media may initially appear daunting. But it’s not rocket science.

Here’s what you need to get the job done: A determination to investigate the backstory of the content, coupled with a healthy level of skepticism and a familiarity with the multitude of free tools that can help establish facts about a video. This chapter will help to equip you with all three.

A first point to understand about verifying user-generated video is that it spreads across social media in a way that makes the version you first see unlikely to be the original. Videos may be spliced, diced and reposted with different context. Important traces from the original video may disappear. Your job is to root out the facts that support or deny what this video purports to show.

As with any story, start with the basic questions: who, what, when, where and why. In this context, the metadata associated with a video can help answer some of these questions by providing you with details about the original source, date and location.

One rule, however, is that one piece of evidence alone is insufficient to verify a video -usually a body of evidence needs to be collected to form a complete picture. Get ready for that adrenaline rush when the puzzle comes together.

Here’s a step-by-step-guide to verifying video from social media.

Provenance

Identifying a video’s provenance is the first step. Sometimes it is obvious that the video belongs to the Facebook or YouTube account where you discovered it. But as detailed in Chapter 3, you always start from the assumption that a video has been “scraped” or duplicated.

Most videos come with a description, tag, comment or some piece of identifying text. Extract useful keywords from this information to begin your search. Acronyms, place names and other pronouns make good keywords. If the description is in a foreign language, paste the text into Google Translate to highlight these keywords.

Search for the earliest videos matching these keywords using the date filter to order results. On YouTube, look directly below the search bar for the Filters menu and select Upload Date, as in the below image. Vimeo, YouKu and other video platforms have similar filters. Scroll through the results and compare video thumbnails to find the earliest version (the thumbnails of original and “scraped” videos usually match).

Another method to find the earliest version of a video is to perform an image search of the video thumbnail using Google Image Search or TinEye (as explained in the previous chapter). This can identify the first instance of video thumbnails and images. The helpfulness of these tools depends on the image quality; a strong contrast in the video and a distinctive color scheme help.

Once you’ve found the source behind the video, contact the source to begin the next step.

Verify the source

It’s time to examine the source the same way we would look at any more-traditional source of information. Indeed, often much more information is available about an online source than a traditional source telephoning a tip line, for example.

Online profiles leave a digital footprint that allows us to examine history and activity. Most platforms enable us to contact uploaders, which is an essential step. Ultimately we seek to engage with the uploader, ask questions and satisfy ourselves that the uploader filmed the footage.

These questions are useful when examining an uploader’s digital footprint:

• Are we familiar with this account? Has the account holder’s content and report age been reli- able in the past?
• Where is this account registered?
• Where is the uploader based, judging by the account history?
• Are video descriptions consistent and mostly from a specific location? Are videos dated?
• If videos on the account use a logo, is this logo consistent across the videos? Does it match the avatar on the YouTube or Vimeo account?
• Does the uploader “scrape” videos from news organizations and other YouTube accounts, or does he upload solely user-generated content?
• Does the uploader write in slang or dialect that is identifiable in the video’s narration?
• Are the videos on this account of a consistent quality? (On YouTube, go to Settings and then Quality to determine the best quality available.)
• Do video descriptions have file extensions such as .AVI or .MP4 in the video title? This can indicate the video was uploaded directly from a device.
• Does the description of a YouTube video read: “Uploaded via YouTube Capture”? This may indicate the video was filmed on a smartphone.

Gathering the answers to these questions helps paint a picture of the source, the source’s online history and the kind of content he shares. From there, it’s important to try to connect that account’s activity to any other online accounts the source maintains. Below are some practices/questions to guide this process.

• Search Twitter or Facebook for the unique video code - are there affiliated accounts? (Every piece of UGC is identified by a unique code that appears in the URL. On YouTube and Facebook, for instance, the code is placed between “v=” and the next “&” in the URL.)
• Are there other accounts - Google Plus, a blog or website - listed on the video profile or otherwise affiliated with this uploader?
• What information do affiliated accounts contain that indicate recent location, activity, reliability, bias or agenda of the account holder?
• How long have these accounts been active? How active are they?
• Who are the social media accounts connected with, and what does this tell us about the uploader?
• Can we find whois information for an affiliated website?
• Is the person listed in local phone directories, on Spokeo, Pipl.com or WebMii or on LinkedIn?
• Do the source’s online social circles indicate proximity to this story/location?

Asking these questions, and answering them, gives us an impression as to the reliability of a source of content. And, importantly, it provides a means to contact the uploader to seek further questions and guidance on the how the video may be used by news organizations.

When speaking to the source, be sure to ask about some of the information you came across. Do the answers match up? If the source isn’t honest with you about information, then you should be extra suspicious of the content.

Locate the video

With the source identified and examined, it’s time to try to verify the content of the video itself. This begins with confirming, or establishing, the location of the video.

Verifying where a video was filmed very much depends on the clues the video presents. A distinctive streetscape, a building, church, line of trees, mountain range, minaret or bridge are all good reference points to compare with satellite imagery and geolocated photographs. Should the camera pan across a business name, this might be listed in online classifieds or a local directory. A street sign might give clues to the precise location. Car registration plates or advertising billboards might indicate provincial details. Sunlight, shadows and the approximate time of day of the event can also be helpful. And if the video contains dialogue, do the accents or dialects fit the circumstances it purports to represent?

The starting point, again, is to examine any text accompanying the video and clues within the video. Home in on the location using Google Maps and try to map the video location. If possible, zoom into Street View to get the camera angle. If Street View is not available, turn on “Photos” in Google Maps’ options and check if geolocated photographs match the video location. Geolocated photos may also be searched using the advanced search features on Flickr, Picasa and Twitter.

If the video is in a foreign language, enter the text into Google Translate and identify the place name. Be aware that Google Translate often mistranslates: for instance, the Arabic for Lattakia in Syria mistranslates as “Protoplasm,” Daraa as “Shield.” Also be aware that various English transliterations of Arabic render names differently: Jidda or Jiddah, for example. By taking the Arabic text for these places and entering it into Google Maps, we’ll find our way to the city. The below image shows searches in Google Translate and Google Maps.

When translating, use the language skills available among your colleagues and contacts. Translating Japanese characters to Korean or Mandarin yields a more accurate translation than Japanese to English. So if you have a Korean or Mandarin speaker in your midst, or can find one quickly, ask her to investigate the translations for you.

Wikimapia is a crowdsourced version of Google Maps in which buildings, suburbs, military sites and other points of interest are outlined and described. This is useful to get context for an area and identify locations, though this information should be corroborated by other information, as it is possible to encounter errors, or deliberately misleading information.

One example of how Wikimapia can be useful came when a day of “civil disobedience” was held in Port Said, Egypt, in February 2013. Demonstrators were filmed marching by the Port Said University’s Faculty of Education, according to one YouTube uploader. The streetscape was difficult to identify on Google Maps amid the densely packed streets of Port Said. However, the Faculty of Education (ةيبرتلاةيلك) is tagged on Wikimapia; finding and examining this reference point confirmed the location of the demonstration, as shown on the next page.

Google Earth is another useful tool, in that it provides a history of satellite images. This is useful when examining older videos where the terrain may have changed.

Google Earth’s terrain view is also valuable when examining terrain and the relative dimensions of buildings. Recently when the team at Storyful was considering a video as evidence supporting a reported Israeli strike on Syria, Google Earth Terrain’s view of mountains north of Damascus verified the location of a YouTube uploader, as you can see in the below comparison.

Verify the date

Confirming the date of videos uploaded from a planned event like a demonstration or politi- cal rally is generally straightforward. Other videos of the same event are likely to exist via news reports, and corroborating pictures are usually shared on Twitter, Facebook, Instagram and other social media sites. Searching these platforms with relevant keywords and hashtags is usually sufficient to discover supporting evidence such as distinctive buildings or street furniture, placards or weather conditions.

However, for more obscure videos, date is generally the most difficult piece of metadata to verify. YouTube videos are time-stamped in Pacific Standard Time (PST) from the moment the upload begins. This led Russia’s Foreign Ministry to cast doubt on videos depicting a chemical weapons attack on Ghouta near Damascus: The videos were uploaded in the early hours of August 21, and therefore were dated on YouTube as August 20. The Foreign Ministry’s ignorance of this prompted it and others to claim the videos were staged and uploaded ahead of the reported time of the attack.

Weather reports alone are insufficient to verify dates, but they help. As previously detailed, Wolfram Alpha provides weather information about a place on a particular date. After Rita Krill uploaded what purported to be amazing video of a lightning strike in her Florida backyard on October 5, 2012, Wolfram Alpha showed that thunderstorms were active in the area.

And searching Twitter for Naples, Florida, on that date showed a local weatherman asking his followers for pictures of storm clouds in Naples. Below is an image of the Wolfram Alpha search and the tweet.

Final checks: What does the video show?

Now it’s time to bring all of your data together and ask the obvious questions: Does the video make sense given the context in which it was filmed? Does anything jar my journalistic instinct? Does anything look out of place? Do clues suggest it is not legitimate? Do any of the source’s details or answers to my questions not add up? Remember, your assumption is that the video is false. Does the evidence confirm or refute that assumption?

When it comes to video, bear in mind that elaborate hoaxes have been, and continue to be, played. Canadian students infamously faked a video of an eagle swooping down in a park in Montreal and picking up a baby. This was debunked by splitting the video into single frames and spotting that the eagle’s shadow was missing in some frames. (More technical people can use video editing software like the free VLC media player or the free Avidemux video editor, or the licensed Vegas Pro editor to split a video into its constituent frames if you have doubts over its construction.)

Abstract

How a small data journalism team in Cuba fights against the lack of data.

Keywords: Cuba, scarce data, data learning, artificial intelligence, data journalism, researcher–journalist collaborations

Postdata.club is a small team. We started as four journalists and a specialist in mathematics and computer science, who, in 2014, decided to venture together into data journalism in Cuba. Until that moment, there was no media outlet that was explicitly dedicated to data journalism in Cuba and we were interested in understanding what the practice entailed.

Today we are two journalists and a data scientist working in our free time on data stories for Postdata.club.

Data journalism does not feature in our daily jobs. Saimi Reyes is editor of a cultural website, Yudivián Almeida is a professor of the School of Math and Computer Science at the University of Havana, and Ernesto Guerra is a journalist at a popular science and technology magazine. Our purpose is to be not just a media organization, but an experimental space where it is possible to explore and learn about the nation we live in with and through data.

Postdata.club lives on GitHub. This is because we want to share not just stories but also the way we do research and investigations. Depending on the requirements of the story we want to tell, we decide on the resources we will use, be they graphics, images, videos or audio. We focus on journalism with social impact, sometimes long-form, sometimes short-form. We are interested in all the subjects that we can approach with data, but, above all, those related to Cuba or its people.

The way we approach our investigations depends on the data that we have access to. Sometimes we have access to public and open databases. With these, we undertake data analyses to see if there may be a story to tell. Sometimes we have questions and go to the data to find answers that could constitute a story. In other cases, we explore the data and, in the process, find interesting leads or questions which may be answered by data we do not yet hold.

Other times—and on more than a few occasions—to support our analysis and investigations, we have to create databases ourselves based on information that is public but not properly structured. For example, to report on the Cuban elections, we had to build databases by combining information from different sources. We started with data published on the site of the Cuban Parliament. This data, however, was not complete, so we complemented it with press reports and information from Communist Party of Cuba websites.

To report on the recently designated Council of Ministers, it was also necessary to build a database. In that case, the information provided by the National Assembly was not complete and we used press reports, the Official Gazette and other websites to get a more comprehensive picture. In both cases, we created databases in JSON format which were analyzed and used for most of the articles we wrote about the elections and the executive and legislative powers in Cuba.

In most cases we share such databases on our website with an explanation of our methods. However, our work is sometimes complicated by the lack of data that should be public and accessible. Much of the information we use is provided by government entities, but in our country many institutions do not have an online presence or do not publicly report all the information that they should. In some cases we went directly to these institutions to request access to certain information, a procedure which is often cumbersome, but important.

For us, one of the biggest issues with the data that we can obtain in Cuba is its outdatedness. When we finally get access to the information we are looking for, it is often incomplete or very outdated. Thus, the data may be available for consultation and download on a website, but the most recent date covered is from five years ago. In these cases we identify other reliable websites which provide up-to-date information or resort to documents in print, scans or human sources.

Collaborations with students and researchers are one of the ways we approach situations where information is missing, outdated or scarce. Since 2017, we have taught a data journalism course to journalism students at the University of Havana School of Communication. Through our exchanges with these future journalists and communication professionals we have learned new ways of working and discovered new ways to access information.

One of the things we do in these classes is to involve students in the construction of a database. For example, there was no single source in Cuba to obtain the names of the people who have received national awards, based on their life’s work, in different areas. Together with students and teachers, we collected and structured a database of the recipients of more than 27 awards since they began to be granted until today. This information allowed us to reveal that there was a gender gap in awarding prizes. Women received these prizes only 25% of the time. With this discovery we were able, together, to write a story that encouraged reflection about gender issues in relation to the national recognition of different kinds of work.

In 2017 we had another revealing experience. This experience helped us to understand that, in many cases, we should not to settle for existing published databases and we should not make too many early assumptions about what is and is not possible. As part of their final coursework, we asked students to form small teams to carry out an investigation. These were composed, in each case, by one of the four members of the Postdata.club team, two journalism students and a student of computer science. One of the teams proposed tackling new initiatives of self-employment in Cuba. Here, these people are called cuentapropistas. What was a few years ago a very limited practice, is now rapidly growing due to the gradual acceptance of this form of employment in society.

We wanted to investigate the self-employment phenomenon in Cuba. Although the issue had been frequently addressed, there was almost nothing about the specificities of self-employment by province, the number of licenses granted per area of activity or trends over time. Together with the students, we discussed which questions to address and came to the conclu- sion that we lacked good data sources. In places where this information should have been posted publicly, there was no trace. Other than some interviews and isolated figures, not much information on this topic was available in the national press.

We thought that the data would be difficult to obtain. Nevertheless, journalism students from our programme approached the Ministry of Labour and Social Security and asked for information about self-employment in Cuba. In a few days the students had a database in their hands. Suddenly, we had information that would be of interest to many Cubans, and we could share it alongside our stories. We had wrongly assumed that the data was not intended for the public, whereas the ministry simply did not have an up-to-date Internet portal.

Coincidentally, the information came into our hands at a particularly convenient moment. At that time, the Ministry of Labour and Social Security decided to limit license issuing for 28 of the activities authorized for non-state employment. We were thus able to quickly use the data we had obtained to analyze how these new measures would affect the economy of the country and the lives of self-employed workers.

Most of our readers were surprised that we were able to obtain the data and that it was relatively easy to obtain. In the end it was possible to access this data because our students had asked the ministry and until today Postdata.club is the online place that makes this information publicly accessible.

Doing data journalism in Cuba continues to be a challenge. Amongst other things, the dynamics of creating and accessing data and the political and institutional cultures are different from other countries where data can be more readily available. Therefore, we must always be creative in looking for new ways of accessing information to tell stories that matter. It is only possible if we continue to try, and at Postdata.club we will always strive to be an example of how data journalism is possible even in regions where data can be harder to come by.

One of the things that I am regularly asked by journalists is how do I get a coder to help me with my project?. Don’t be deceived into thinking this is a one-way process; civic-minded hackers and data-wranglers are often just as keen to get in touch with journalists.

Journalists are power-users of data driven tools and services. From the perspective of developers: journalists think outside the box to use data tools in contexts developers haven’t always considered before (feedback is invaluable!) they also help to build context and buzz around projects and help to make them relevant. It is a symbiotic relationship.

Fortunately, this means that whether you are looking to hire a hacker or looking for possible collaborations on a shoestring budget, there will more than likely be someone out there who is interested in helping you.

So how do you find them? Says Aron Pilhofer from the New York Times:

You may find that your organzation already has people with all the skills you need, but they are not necessarily already in your newsroom. Wander around, visit the technology and IT departments and you are likely to strike gold. It is also important to appreciate coder culture, come across someone who has a computer that looks like the one here…

Here are a few more ideas:

Once you’ve found a hacker, how do you know if they are any good? We asked Alastair Dant, the Guardian’s Lead Interactive Technologist, for his views on how to spot a good one:

How To Find Your Dream Developer

The productivity difference between a good and a great developer is not linear, it’s exponential. Hiring well is extremely important. Unfortunately, hiring well is also very difficult. It’s hard enough to vet candidates if you are not an experienced technical manager. Add to that the salaries that news organzations can afford to pay, and you’ve got quite a challenge.

At Tribune, we recruit with two angles: an emotional appeal and a technical appeal. The emotional appeal is this: Journalism is essential to a functioning democracy. Work here and you can change the world. Technically, we promote how much you’ll learn. Our projects are small, fast and iterative. Every project is a new set of tools, a new language, a new topic (fire safety, the pension system) that you must learn. The newsroom is a crucible. I’ve never managed a team that has learned so much, so fast, as our team.

As for where to look, we’ve had great luck finding great hackers in the open government community. The Sunlight Labs mailing list is where do-gooder nerds with shitty day jobs hang out at night. Another potential resource is Code for America. Every year, a group of fellows emerges from CfA, looking for their next big project. And as a bonus, CfA has a rigorous interview process — they’ve already done the vetting for you. Nowadays, programming-interested journalists are also emerging from journalism schools. They’re green, but they’ve got tons of potential.

Lastly, it’s not enough to just hire developers. You need technical management. A lone-gun developer (especially fresh from journalism school, with no industry experience) is going to make many bad decisions. Even the best programmer, when left to her own devices, will choose technically interesting work over doing what’s most important to your audience. Call this hire a news applications editor, a project manager, whatever. Just like writers, programmers need editors, mentorship and somebody to wrangle them towards making software on deadline.

Written by Malachy Browne

One of the iconic videos of the tragic 2013 Boston bombings was filmed by an athlete running her final mile of the marathon. As she approached the finish line on Boylston Street, the second bomb detonated meters ahead. It was a compelling video, but we needed to verify it.

One photo showing the moment of the blast was posted by Boston journalist Dan Lampariello (below), a member of one of our pre-curated Twitter lists, and someone familiar to Storyful. Lampariello’s tweet was geolocated to Boylston Street; this information, which came from a reliable source, helped to confirm the location of the explosion. It also gave us a reference point to use with what was shown in the runner’s video.

Google Street View of Boylston street (below) confirmed both Dan Lampariello’s photo and the athlete’s point of view as she approached the finish line. Indeed, some of the athletes filmed in the video are seen in Lampariello’s photo, upon close inspection.

That process confirmed the content of the video. Finding the original source of this video was less straightforward.

The video itself was uploaded to a YouTube account with no giveaway details and an obscure username, NekoAngel3Wolf. Searching Twitter for the unique video code led us to someone sharing it under the handle NightNeko3, again with no personal details. The “Neko” reference in both profiles suggested they were affiliated.

Searching for similar social profiles, we found a Pinterest account also registered as NightNeko3, giving the real name Morgan Treacy. Our team at Storyful quickly located a Facebook account for Morgan Treacy, a teenager whose posts were geolocated to Ballston Spa in New York State.

Morgan described the video on Twitter as her mother’s perspective of the explosion. Knowing that a prestigious marathon like Boston’s would likely track athlete times, we checked the surname “Treacy” on Boston Athletic Association’s registrant page. A single result was returned - Jennifer Treacy, age 45-49, from New York State. Jennifer Treacy’s time split shows her passing the 40 kilometer mark at 2:38 p.m. but failing to cross the finish line 2 kilometers later. Jennifer was averaging 10 minutes per mile, placing her in the vicinity of the blast at 2:50 p.m., when the bombs exploded.

The social people search website Spokeo.com gave us an entry for Jennifer L. Treacy, 47, with an address at Ballston Spa, New York. LinkedIn also gave us a profile for Jennifer Treacy from Ballston Spa, who is employed by the New York State Department of Health.

One final piece of evidence confirmed our investigation. A man named Gerard Quinn is a Facebook friend of Morgan Treacy, who we were now almost 100 percent sure was Jennifer’s daughter. Quinn previously commented on family videos posted by Morgan. So there was a link between him and the family. We saw on Quinn’s Facebook profile (below) that he had expressed pride that his niece, Jennifer, was running the Boston marathon. He’d linked to her marathon map and time splits. He also later commented on Facebook that Jennifer was OK after the blast and on her way home.

A public telephone directory produced a phone number that allowed us to speak directly to Jennifer Treacy. She confirmed the video was hers and that news organizations were permitted to use it. She had also informed law enforcement agencies of the video, she said.

In summary, all of the information supporting the veracity of this video was available online via free tools - location information, corroborating accounts of the event, the uploader’s digital history and the owner’s contact details. Familiarity with these tools allowed us to verify the video in around 10 minutes.

Charlotte Godart is an investigator and trainer for Bellingcat. Before Bellingcat, she was at the Human Rights Center at UC Berkeley, working within its Investigations Lab, teaching students to conduct open-source research on global conflicts for international humanitarian entities.

Johanna Wild is an open-source investigator at Bellingcat, where she also focuses on tech and tool development for digital investigations. She has an online journalism background and previously worked with journalists in (post-)conflict regions. One of her roles was to support journalists in Eastern Africa to produce broadcasts for the Voice of America.

In late August 2019, Benjamin Strick, a Bellingcat contributor and BBC Africa EYE investigator, was analyzing tweets spreading the hashtags #WestPapua and #FreeWestPapua when he noticed accounts exhibiting abnormal behavior. These accounts were all spreading Indonesian pro-government messages at a moment when the conflict in West Papua was gaining international visibility: A local independence movement had taken to the streets to fight for freedom from Indonesian control, leading to violence between the Indonesian police and protesters.

The accounts Strick saw exhibited multiple odd similarities. Soon, he would realize that these were the early indicators of coordinated inauthentic behavior. But at first, he started by noticing the small stuff.

For one, many of the accounts had stolen profile pictures. Take this account for instance, which claimed to be of someone named Marco:

Using Yandex’s reverse image search tool, Strick found that the account’s profile picture had been previously used on other websites under different names. None of the accounts using the photo were for a real person named “Marco.” This proved that the accounts were, at the very least, misleading about their true identities.

Beyond faking their identities, Strick also found the accounts published similar or even identical content while often retweeting one another. Even more striking was that some of them showed precise synchronization in the timecode patterns of their tweets. For example, @bellanow1 and @kevinma40204275 mostly published their tweets at minute 7 or minute 32 of any particular hour.

It’s unlikely that a human would adopt this kind of tweet rhythm. This synchronization across multiple accounts, combined with their misleading photos, suggested the accounts were not linked to real identities, and could be automated. By analyzing suspicious account patterns such as these, Strick eventually concluded that the accounts were part of a pro-Indonesian Twitter bot network that was spreading one-sided and misleading information about the conflict in West Papua. (You can read more about the larger network these accounts were part of in the chapter 11b case study, “Investigating an Information Operation In West Papua.”)

What’s a bot? The answer is more complicated than you might think

The West Papua case is far from being the only information operation to use social bots. Other operations have been much more widely publicized and criticized, although at their core they contain similarities in how they operate.

A bot is a software application that can automatically perform tasks assigned to it by humans. Whether a bot does good or bad completely depends on the intentions of its “owner.”

The bots most often referred to in public debates are social bots, active on social networks including Facebook, Twitter and LinkedIn. On these platforms, they can be used to spread specific ideological messages, often with the aim to make it look as if there is a groundswell of support for a particular topic, person, piece of content or hashtag.

Social media bots tend to fall into three main categories: the scheduled bot, the watcher bot and the amplifier bot. It’s important to know which kind of bot you’re interested in because each type has a specific purpose. With each purpose comes a different language and communication pattern. In the context of disinformation, we’re most interested in looking into the amplifier bot.

The amplifier bot exists to do exactly what it sounds like: amplify and spread content, with the goal of shaping online public opinion. It can also be used to make individuals and organizations appear to have a larger following than they really do. Its power comes in numbers. A network of amplifier bots can attempt to influence hashtags, spread links or visual content, or gang up to mass spam or harass an individual online in an attempt to discredit them or to make them seem controversial or under siege.

By working together in large numbers, amplifier bots seem more legitimate and therefore help shape the online public opinion landscape. Amplifier bots that spread disinformation do it mainly through hashtag campaigns or by sharing news in the form of links, videos, memes, photos or other content types. Hashtag campaigns involve bots constantly tweeting the same hashtag, or set of hashtags, in coordination. The goal is often to trick Twitter’s trending algorithm into adding a specific hashtag to the trending topics list. An example is “#Hillarysick,” which was propagated widely by bots after Hillary Clinton stumbled in September 2016, shortly before the presidential election. (It’s also important to note that hashtag campaigns don’t require bots, and can be more effective without them. See this investigation of human “hashtag mills” in Pakistan from Dawn.)

Purchasing and creating bots is relatively easy. Countless sites will sell you your own bot army for just a couple of hundred dollars or even less. But a sophisticated, humanlike botnet is much harder to create and maintain.

How to recognize bots

Developers and researchers have created many tools to help assess whether an account might be automated. These tools can be useful in gathering information, but a score from one tool is by no means definitive and should never form the sole basis of any reporting or conclusion.

One of the most well-known tools is Botometer, created by researchers at Indiana University. Based on various criteria, it calculates a score for how likely it is that a Twitter account and its followers are bots.

For Reddit, Jason Skowronski has created a real-time dashboard. After you set it up for a chosen subreddit, it tries to assess whether the comments were made by bots, trolls or humans.

While there are exceptions, most publicly available bot detection tools have been created for Twitter. The reason is that many social networks — including Facebook — restrict their APIs (application programming interfaces) in a way that prevents the public from analyzing and using their data to create such public tools.

As noted earlier, bot detection tools are a great starting point but they should not be your sole evidence. One reason for their varying degree of accuracy is there is simply no universal list of criteria for recognizing bots with 100% certainty. There’s also little agreement about how to classify something as a bot. Researchers at the Oxford Internet Institute’s Computational Propaganda Project classify accounts that post more than 50 times a day as having “heavy automation.” The Atlantic Council’s Digital Forensics Research Lab considers “72 tweets per day (one every ten minutes for twelve hours at a stretch) as suspicious, and over 144 tweets per day as highly suspicious.”

It can often be challenging to determine whether a disinformation campaign is conducted by social bots or by humans who are motivated or paid to post large amounts of content about a specific topic. The BBC, for instance, found that accounts who posted similar Facebook messages amplifying favorable content about Boris Johnson in November 2019 were managed by people who pretended to be social bots.

You might also encounter cyborgs, social media accounts that are partly automated and partly managed by humans, which display a combination of natural and inauthentic behavior. Journalists must avoid falsely labeling suspicious accounts as bots without proper evidence and analysis, as a mistaken accusation can undermine your credibility.

One way to deal with these different types of bots, cyborgs and hyperactive human accounts is to focus your investigation on monitoring all inauthentic or bot-like behavior, instead of trying to identify only one type of suspicious account.

For example, Bot Sentinel provides a publicly available database containing (U.S.) Twitter accounts that exhibit suspicious behavior. Their creators decided to collect “accounts that were repeatedly violating Twitter rules” instead of specifically searching for social bots.

Steps to investigate inauthentic behavior

In general, we suggest the following approach for identifying inauthentic and potentially automated behavior on social networks:

1. Manually check the accounts for suspicious behavior.

2. Combine this with the use of tools or more technical network analyses.

3. Investigate their activity, content and network of other accounts they interact with. Combine this with traditional investigation techniques, such as trying to contact them or people they claim to know.

4. Consult with outside experts who specialize in bots and inauthentic activity.

To learn how to manually assess suspicious accounts, it’s important to understand the typical warning signs of automated accounts on Twitter, or other social networks.

Every social media bot needs an identity. Bot creators want to make their accounts appear as convincing as possible, but it takes time to set up and maintain credible-looking profiles, in particular if the goal is to run a large bot network. The more accounts someone has, the more time-consuming it is to create and manage them in a way that makes them seem authentic. This is where these accounts slip up. In many cases, their creators do the bare minimum to establish a profile, and a good investigator can detect this.

Here are a few things to look for:

No real profile picture

A stolen profile picture (as seen in Benjamin Strick’s West Papua investigation) or no profile picture at all can be an indicator of inauthenticity. Since bot creators want to create many accounts at once, they have to obtain a collection of photos and often copy them from other websites. However, doing so creates inconsistencies. For instance, an account with the profile photo of a male but a username implying that a female is the owner of the account could be a signal that something isn’t right. To get around this issue, many bot creators choose cartoons or animals as profile pictures, but again this tactic becomes another pattern to use to detect inauthentic or bot accounts.

Automatically created usernames

Next, look out for names and usernames. Every Twitter handle is unique, which means the username you want is often already taken. This is an inconvenience to the average person, but becomes a real challenge when you’re trying to create 50, 500 or 5,000 accounts in a short period of time.

Bot creators often deploy a strategy to help them easily find unused usernames. Scripts with criteria like the following are used to automatically create usernames:

When you notice several Twitter accounts with handles consisting of the same number of characters and digits, you can manually search for more accounts with that pattern in each of the accounts’ followers list to potentially identify a network.

In this example, the accounts have something else in common: They all were created in September 2019. When combined with other signals this can be an indicator that the accounts were all done at the same time by the same person.

Account activity does not fit age

You should become even more suspicious if a new account already has a relatively large number of followers or if it has published a large number of tweets within a short time. The same applies if an older account has very few followers despite being very active.

If you come across such an account, analyze the account’s tweet activity more deeply. Take the number of tweets located at the top of the page, and divide this by the number of days the account has been active. For example, take an account that has 3,489 tweets as of Nov. 11, 2019, and was created on Aug. 15, 2019. Divide 3,489 by 89 (the days it’s been active), and you get 39.2 tweets per day.

Looking at the tweets made over the lifetime of the account, does the number seem too high, unrealistic or not maintainable?

Suspicious tweet patterns

Another element to examine is tweet rhythm. Humans might show slight preferences for the days and times they usually tweet, but it is unlikely that a person posts consistently only on Monday, Tuesday and Wednesday and is completely silent on all other days of the week over a long period of time.

If you want to see these patterns visualized for one specific account, check out the account analysis tool built by Luca Hammer:

Visualization as part of your investigation

To get a better understanding of the activity of a whole bot network, you can use a visualization platform like Gephi. Bellingcat contributor Benjamin Strick used this tool to analyze the connections between Twitter accounts belonging to a pro-Indonesian bot network.

By looking at the visual representation of the connections between a large number of Twitter accounts, he noticed that the structure on the left side of the picture (in red) stood out.

By zooming in on this area, he could see which Twitter accounts were part of this specific structure.

Each red circle represents a Twitter account and the lines are the relationships between them. Usually, smaller accounts are arranged around a bigger circle in the middle, which means that they all interact with the influential account. The accounts in the structure above, however, did not interact in that way with one another. This led Strick to analyze those abnormal account’s behavior.

The future of social bots: Can we out-trick them?

The technology behind social bots has become much more advanced in the last few years, allowing these small software applications to become more adept at simulating human behavior. We are getting to the point where people are predicting that artificial users could engage in sophisticated online communications without their human counterparts realizing that they’re actually having a long conversation with a bot.

However, as of now there is no proof that high-level, machine-learning-empowered social bots exist or are being deployed. For now, it seems that many disinformation campaigns are currently still receiving support from less-complex amplifier bots.

“I don’t think that there are many sophisticated social bots out there that are able to have real conversations with people and to convince them of certain political positions,” said Dr. Ole Pütz, a researcher for the project “Unbiased Bots that Build Bridges” at Bielefeld University in Germany.

According to him, the best way to help the public recognize inauthentic behavior on social networks is to use a detection method that catalogs and weighs all the factors that make an account suspicious. As an example, he says, “This account uses a script to retweet news, it automatically follows others, and that one never uses speech patterns that humans would normally use.”

For now, a methodical analysis of account behavior, content, interactions and patterns remains the best approach for identifying inauthentic behavior.

In our case study chapter, we provide a more in-depth and technical explanation of how we analyzed the different factors in a suspicious Twitter network related to the Hong Kong protests.

Written by Flor Coelho

Abstract

Using civic marathons and the open-source platform Vozdata to collaborate with readers, universities and NGOs around large data-driven investigations.

Keywords: civic marathons, crowdsourcing, investigative journalism, open source, researcher–journalist collaborations, open data

At La Nación we have produced large data-driven investigations by teaming up with our readers. This chapter takes a look behind the scenes at how we have organized reader participation around some of these projects, including through setting goals, supporting investigative communities, and nurturing long-term collaborations with our readers and other external organizations and partners.

In such projects often our goal is to tackle the “impossible” by using technology to facilitate large-scale collaborations, enabling users to engage with investigative journalism and the process of making official data public.

For example, we spent around five years transcribing 10,000 PDFs of Senate expenses, two years listening to 40,000 intercepted phone calls and a couple of months digitizing more than 20,000 hand-written electoral forms.1

For these kinds of crowdsourcing initiatives, we relied on the online collaborative platform Vozdata. The platform was inspired by The Guardian’s MPs’ expenses and ProPublica’s “Free the Files'' crowdsourcing campaigns and was developed with the support of Knight-Mozilla OpenNews and CIVICUS, a global alliance of civil society organizations and activists. The software behind Vozdata was open-sourced as Crowdata.2

Organizing Participation

For these projects our collaborators were mainly journalism students, civic volunteers, transparency NGOs and retired citizens. They have different motivations to participate depending on the project. These may include contributing to public interest projects, working with our data team and getting to know other people at our meetups.

Vozdata has teams and live ranking features. We have been exploring how these can enhance participation through “gamification.” We had excellent results in fostering civic participation in this way around Argentina’s national holidays. Participation in the construction of collaborative databases is mostly undertaken remotely (online).

But we have also encouraged users to participate in “offline” civic events held at La Nación or during hackathons at various events. Sometimes we have built open (i.e., freely reusable) databases with journalism students at partner universities.

While hackathons are events that usually take one or two days, our online marathons can continue for months. The progress bar shows how many documents have been completed and the percentage that remain to be completed.

Setting Big Goals

The main role of collaborators in the Senate Expenses and Electoral Telegrams projects was to gather specific structured data from the documents provided. This involved over a thousand unique users. As well as extracting these details, readers also had the opportunity to flag data as suspicious or unacceptable and leave a comment with additional information.

The latter feature was rarely used. When you have a deadline to finish a crowdsourcing project, you may not reach your target. That happened to us in the Electoral Telegrams project. The election day was approaching and we needed to publish some conclusions. While some provinces reached 100%, many had only completed 10% to 15% of the files, which we acknowledged when we published.

Supporting Investigative Communities

For Prosecutor Nisman’s 40,000 files investigation, we worked with a trusted network of a hundred collaborators. Many audio files related to private conversations (e.g., family dialogues) held by the Iranian agent whose phone was tapped by a federal court. A group of six volunteers got really deep into the investigation. We created a WhatsApp group where anyone could suggest leads and curiosities.

One of our volunteers resolved a mystery that kept us busy for a couple of months. We had flagged several conversations where two people talked in code using nicknames and numbers (e.g., “Mr. Dragon, 2000”). Many volunteers had heard and transcribed such recordings. We thought about making a separate database to analyze the code behind them. One day, a volunteer discovered that the conversations were about betting on horse races! A quick Google search confirmed many names as racing horses.

You always have power users. But, depending on the scale of the project, many volunteers collaborating with a few documents each usually exceed the “superuser contribution.”

Nurturing Collaborations

Our advice for journalists and organizations who want to involve their readers in data investigations is to appoint a dedicated community manager to organize and deliver communications through collaborative spreadsheets (e.g., Google Sheets), mailing lists and social media.

Large collections of documents can be a good place to start: The learning curve is fast and participants feel part of something bigger. It’s also valuable to support collaborators with video tutorials or contextual introductions in-house at your organization or at dedicated events.

When we won an award related to these collaborative projects, we hosted a breakfast to share the prize with the volunteers. These are long-term relationships with your readers, so we made sure to dedicate time and energy to meeting up at events, visiting universities, giving interviews for student projects and so on.

Regarding partnerships with universities, professors usually act as nodes. Every year they have a new class of students who are usually eager to team with us in collaborative projects (plan for this in advance!).

Transparency NGOs can also demonstrate the benefits of these projects. In our platform, every task can be registered, so they can easily showcase projects and media recognition for their donors.

When publishing outputs and stories, we recommend acknowledging the collaboration process and participant organizations in every platform (print, online and social media) and in mailouts. Emphasizing the collective character of such projects can send a stronger message to those who we want to hold accountable.

Conclusion

To make data with readers it is vital to allocate time and resources to engage with your community, deal with requests, analyze outputs, enjoy interactions and participate in events.

Volunteers classify documents because they think a project matters. For governments and those being reported on it is a sign that the project is not only a press concern, but also affects civil society. Through such projects, participants can become passionate advocates and online distributors of the content.

Footnotes
1. blogs.lanacion.com.ar/projects/data/argentina, blogs.lanacion.com.ar/projects/data/prosecutor, blogs.lanacion.com.ar/projects/data/vozdata

2. blogs.lanacion.com.ar/projects/data/vozdata--ii-civic, theguardian.com/news/datablog/2009/jun/18/mps-expenses-houseofcommons, propublica.org/series/free-the-files, github.com/crowdata/cr...

In March 2010, Utrecht based digital culture organzation SETUP put on an event called ‘Hacking Journalism’. The event was organised to encourage greater collaboration between developers and journalists.

‘We organize hackathons to make cool applications, but we can’t recognise interesting stories in data. What we build has no social relevance’, said the programmers. ‘We recognize the importance of data journalism, but we don’t have all the technical skills to build the things we want’, said the journalists.

Working for a regional newspaper, there was no money or incentive to hire a programmer for the newsroom. Data journalism was still an unknown quantity for Dutch newspapers at that time.

The hackathon model was perfect. A relaxed environment for collaboration, with plenty of pizza and energy drinks. RegioHack was a hackathon organised by my employer, the regional newspaper De Stentor, our sister publication TC Tubantia and Saxion Hogescholen Enschede, who provided the location for the event.

The setup was as following: everyone could enlist for a 30-hour hackathon. We provided the food and drink. We aimed for 30 participants, which we divided into 6 groups. These groups would focus on different topics, such as crime, health, transport, safety, ageing and power. For us, the three main objectives for this event were as follows:

Before the event, TC Tubantia had an interview with the widow of a policeman who had written a book on her husband’s working years. She also had a document with all registered murders in the eastern part of the Netherlands, maintained by her husband since 1945. Normally, we would publish this document on our website. This time, we made a dashboard using the Tableau software. We also blogged about how this came together on our RegioHack site.

During the hackathon, one project group came up with the subject of development of schools and the ageing of our region. By making a visualization of future projections, we understood which cities would get in trouble after a few years of decline in enrolments. With this insight, we made an article on how this would affect schools in our region.

We also started a very ambitious project, called De Tweehonderd van Twente (in English, The Two Hundred of Twente) to determine who had the most power in our region and build a database of the most influential people. Through a Google-ish calculation — who has the most ties with powerful organizations — a list of influential people will be composed. This could lead to a series of articles, but it’s also a powerful tool for journalists. Who has connections with who? You can ask questions to this database and use it in our daily routine. Also, this database has cultural value. Artists already asked if they could use this database when finished to make interactive art installations.

After RegioHack, we noticed that journalists considered data journalism as a viable addition to traditional journalism. My colleagues continued to use and build on the techniques learned on the day to create more ambitious and technical projects such as a database of the administrative costs of housing. With this data, I made an interactive map in Fusion Tables. We asked our readers to play around with the data and crowdsourced results (here, for example). After a lot of questions on how we made a map in Fusion Tables, I also recorded a video tutorial.

What did we learn? We learned a lot, but we also came along a lot of obstacles. We recognized these four:

What now? Our first experiences with data journalism could help other journalists or programmers aspiring the same field of work and we are working to produce a report.

Also, we are considering how to continue RegioHack in a hackathon form. We found it fun, educational and productive and a great introduction to data journalism.

But for data journalism to work, we have to integrate it in the newsroom. Journalists have to think in data, in addition to quotes, press releases, council meetings and so on. By doing RegioHack, we proved to our audience that data journalism isn’t just hype. We can write better informed and more distinctive articles, while presenting our readers different articles in print and online.

Written by Malachy Browne

In March 2011 a graphic video surfaced on YouTube that depicted what was claimed to be the killing of at least six women by Ivorian security forces (FDS) during a protest in Abobo. The demonstration occurred during a period of unrest when President Laurent Gbagbo clung to power after his defeat in presidential elections the previous November.

At the behest of a client, Storyful set about verifying the video two years after it happened. The video shows a large group of women chanting “ADO” (a reference to Alassane Dramane Ouattara, Gbagbo’s rival). Then, at the 3:32 mark, armored personnel carriers come into view and large-caliber rounds are fired. Several people appear to be fatally wounded. At the time, some Ivorians claimed the injuries were staged. The country’s then defense minister cast doubt over the video and Gbagbo supporters claimed the video was a “fake” in YouTube reconstructions (here and here).

Verifying video in a breaking news scenario is in some respects easier than this form of retrospective investigation. Information that corroborates or debunks a video is more accessible in the recent timeframe; information related to an older event is often hidden deep within social networks. Archival search is either challenging or not possible.

With those limitations in mind, here’s how I worked to try to verify the video.

Gather context on the event

Unfamiliar with the details of the reported massacre, I searched Google for “Women killed Gbagbo March 3 2011.” This returned several reports (here and here) describing the ap- proximate location and the sequence of events. This search also returned a statement about the event made by the country’s then defense minister, who claimed the scenes were staged.

Importantly, these reports also provided keywords I could use to run a more focused search. Using these terms for historical search on Twitter and YouTube, I unearthed eyewitness accounts and UGC. (Always try to put yourself in the shoes of the uploader and imagine how she would tag and describe video and other information.)

Location

According to reports, the demonstration and shooting happened at a roundabout in the vicinity of Abobo, a northern district of Abidjan. Specifically, one report located it at a major junction/roundabout on Autoroute d’Abobo, adjacent to the area known as Abobo Gare. A witness in the report described the security forces passing by a roundabout, doubling back and opening fire on the women “before heading back to Adjamé.” Adjamé lies south of Abobo, giving us a lead on the direction of traffic.

According to a contemporaneous report published in Le Patriot on March 8, demonstrators gathered “at the roundabout intersection of Banco” (mapped below). Searching a local forum shows that the roundabout was the site of previous such demonstrations.

Google Maps shows two major roundabouts. One of them, Carrefour Banco, lies at the southern end of Abobo, toward Adjamé. This fit with the previous report, so I used it as my starting point.

The position of street lights and traffic lights, the alignment of palm trees and deciduous trees filmed in the video from 4:00 onward line up with the satellite view of Banco Carrefour’s north-western corner, as shown in the above white circles. The large building with two prominent protrusions atop the roof (circled in red) also aligns with a building we see in the distance as the convoy of security vehicles disappears from view. This matches the direction of traffic evident in the satellite image above, and the account given by an eyewit- ness of the vehicles driving south toward Adjamé.

One piece of video evidence (above), however, did not match the satellite imagery. We counted three large deciduous trees as the convoy entered the roundabout; Google Maps shows just two such trees. The video was filmed in 2011 and the satellite images were dated 2013, so perhaps a tree was cut down. So we looked through historic satellite images on Google Earth. Images from 2009 show three large deciduous trees stood at this corner of the roundabout.

The third, missing tree from the 2013 satellite imagery is outlined in the above image. (It has been flipped 180 degrees from north to south). Judging by this view, we can see that the camera position was directly across the road. I later spoke with a reputable source known to Storyful who is familiar with the video, and who had visited Abobo to report on the “massacre.” The source confirmed this was the camera angle.

Date

The date of the shooting is corroborated by several independent reports and videos shared on social media. These are found retrospectively through a variety of searches: on Twitter, on Topsy or Topsy Pro (which allows a date range to be set), and on YouTube with results ordered by upload date.

Some of the steps I followed:

• I used historical Twitter search to generate leads by scrolling back to results from March 3, 2011, onwards.
• I examined Tweets and questions about the event and found this and this reply. These sources are potential witnesses, or people who could identify witnesses. The first source lists her location as Cocody, Abidjan, and the second one as Abidjan.
• I also located this person, who uploaded video from Abobo and previous RHDP rallies. Checking other Twitvids on his account leads to a video uploaded on the day of the protest.
• I looked further at his Twitter timeline and found other references to RHDP on that day. That led me to other links, such as this news report of the event. It included a photo credited to Reuters that showed victims matching those in our video.
• Running a Google Image Search on the photo confirmed it wasn’t used prior to March 3. However, the results also show that a Guardian article credited AFP/Getty Images and not Reuters. This meant a credible photographer was on the ground at the event.
  

I dug further into the photo, shown below.

The image is consistent with the picture of the victim at 5:30 in the lead video. The victim is covered by garments and green leaves used by many of the demonstrators. Note the tight, dark blue T-shirt worn by the victim and the distinctive garment with a square pattern of red, orange, white and dark lines, shown over the page in a close-up.

France 24 Observateurs was also provided with photos from the event by sources in Abid- jan. We at Storyful confirmed this with France 24.

Other searches uncovered a photo-diary published here by an Agence France-Presse journalist, Issouf Sanogo. Sanogo interviewed a woman named Sirah Drane, who says she helped organize the demonstration on March 3. Drane says she was holding a megaphone to address the large crowd that had gathered at a traffic circle in Abobo. A woman matching this description is seen in the video.

The video correlates with three other videos of the event. These videos were documented by Storyful at the time, and could be found by searching YouTube using search terms identified earlier.

The first video was uploaded on the day of the shooting to an Ivory Coast-registered YouTube account which was created specifically to upload the video. There is no further activity on the account to provide information regarding the source. The same wounded women are filmed in the video, as is the distinctive square building in the background.

A second video was uploaded to another Ivory Coast-registered YouTube account on the morning of March 4 at 09:06:37 GMT. The uploader describes it as “several women killed” at the “RHDP demonstration yesterday,” meaning March 3.

None of these videos or corroborating photos exist before March 3, suggesting to a high degree of certainty this was the date of the event.

Original uploader

The video itself was uploaded to YouTube on March 4, 2011. It‘s the earliest such video found on YouTube. However, it‘s highly likely the video originated from a Facebook account or elsewhere and was scraped onto this YouTube account.

The YouTube account is registered in the United States and is linked to a defunct website, onemendo.com. The account appeared to be operated by someone with connections to Jamaican emigrants living in New York or New Jersey because the account contained promotional material for a local club, DanceHallReggae.com.

Videos from around that time on an affiliated Vimeo account indicate they are based in Rochester, New York. An affiliated Facebook account also posts links to music by Jamaican DJs. It gives no further clues as to the origins of the video and did not post a link to it on March 3, 2011. Videos of a Senegalese soap opera were also posted to the YouTube account.

Is the video authentic?

The evidence above confirms the location and establishes the date of the video as highly likely to be March 3. However, to the central point: Does the video show women protesters being shot dead by the FDS on that day?

Claims have been made that the killing is staged and bodies were placed on the street after the security forces drive past. These serious questions warrant investigation.

In this statement, Gbagbo’s defense minister, Alain Dogou, referred to the emergence of this amateur video on March 4. He said a woman was instructed to “lay down, lay down,” (and we do hear this said in the video). Dogou said it is “difficult to say” that the video is from the location reported by journalists. (Of course, we have confirmed the location.) He also said international journalists were not covering the protest because they were attending a news conference by UNOCI, or another event related to the Council of Ministers. Finally, he acknowledged that a Women’s March did take place in Abobo on this date.

Serious questions that arise:

• Why did the camera point away from the wounded for so long as the convoy entered the roundabout?
• Would all the victims be shot within meters of one another?
• Would they all fall face down as they have in the video?
• Their faces are quickly obscured by garments - why is this?
• A bloodied woman is told to “lay down, lay down” in the video, as described by Defense Min- ister Dogou. Why is this? Is this out of concern for her poor condition, or to stage an injury?
• The “massacre” creates a frenzy of emotion in the video; is this real?
  Or were other protesters duped by or complicit in a staged “massacre”?

Several witnesses give convincing accounts that injuries did result from the reported massacre. A doctor from South Abobo Hospital is quoted on page 63/64 in this Human Rights Watch report. The doctor reported seeing victims from the shooting:

(The video does appear to show a victim whose head was blown apart.)

A New York Times report quoted two named witnesses as follows:

“The forward tank started firing,” said one Abobo resident, Idrissa Diarrassouba. “Right away six women were killed. I was right there, beside them. They just fell.”

“There was a burst of machine-gun fire,” [the witness, Idrissa Sissoko] said. He also spoke of seeing six women being shot. “I saw six bodies lying there, suddenly,” he said.

According to this report, a military source told a Reuters journalist that the shooting was an accident resulting from the nervousness of the security forces following previous clashes.

Conclusion

We can say that the date and location are verified to a high degree. The original source is not, and we therefore did not get the opportunity to speak to the person who filmed the footage.

Ultimately, though, does the video show what it claims?

This we cannot determine to 100 percent satisfaction from a distance, and with the material that’s been gathered. Along with being able to contact and interview the uploader, it would be important to gather additional firsthand testimony from witnesses, doctors who treated victims, and the families of the reported victims. To identify those victims we could attempt a more detailed investigation of the first video, splitting it frame by frame at the key moments of the shooting to try and find ways to the identify victims, and then to track down their survivors.

Even with all of the corroborating facts and information I was able to marshal, the verdict is still out on this video.

Written by Crina-Gabriela Boroş

Abstract

Read this first before running a survey for accountability journalism: dos, don’ts and how to handle imperfect circumstances.

Keywords: statistics, data journalism, surveys, accountability

Is an issue anecdotal or systematic? You’re attempting to discover this when you realize there is not any tabular data—a fancy phrase that simply means information supplied in rows and columns. What should you do?

What is data, anyway? There are many nerdy definitions floating around, some of which are intimidating.1 Let’s trade them for the simple concept of “information.” And as you gather it, in any shape or form, you need to be able to find patterns and outliers. This means that you have to have a considerable amount of systematically gathered raw material that documents an issue according to a specific method (think fill-in forms). Whether you use a spreadsheet, a coding environment, an app, or pen and paper, it does not matter.

Sometimes, thoughts, feelings or past intimate experiences trapped in people’s hearts and minds can be articulated as data. One method of harvesting this precious information is to design a survey that would gather and order such feelings and experiences into a table, archive or a database that nobody else but you has access to.

For instance, the Thomson Reuters Foundation (TRF) undertook a project reporting on how women in the world’s largest capitals perceived sexual violence on public transport affects them.2 It was a survey-driven effort to raise awareness of the issue, but also to compare and contrast (the stuff stats do).

To deliver this spotlight, we went through several circles of Hell, as there are rigorous conventions that social scientific methods, like surveying, require, even when imported by journalists into their practice.

Here are a few main polling rules that journalists would benefit from knowing, but often don’t receive training for.

-Respondents cannot be handpicked. In order to be considered “representative” a pool of respondents would conventionally include people from all social categories, age groups, education levels and geographical areas that we have to report on. According to established methods, samples of the population under study need to be representative.

-The selection of respondents needs to be randomised—meaning everyone has the same chance of having their name drawn from a hat. If you’re conducting a poll and speaking to whomever is closest to hand without any criteria or method, there is considered to be a risk of producing misleading data, especially if you are aiming to make more general claims.

-The number of people taking a survey must also reach a certain threshold for it to be representative. There are helpful online calculators, like those provided by Raosoft, Survey Monkey or Survey Systems.3 As a rule of thumb: Keep the confidence level at 95% and the margin of error no bigger than 5%. Answer options must allow respondents to not know or not be certain. When reporters follow these basic rules, their findings are close to unattackable. At the time of the TRF public transport safety research, our polling methodology stuck to the conservative rules of social sciences. Our subject addressed such a common human experience that speaks volumes about how societies function, that a UN agency offered to join in our effort. An honour, but one which, as journalists, we had to decline.

If you like the sound of this, it’s time to take a stats course.

Sometimes rigorous polling is unrealistic. This doesn’t always mean you shouldn’t poll.

While there are established methods for surveying, these don’t exhaust what is possible, legitimate or interesting. There may be other ways of doing polls, depending on your concerns, constraints and resources.

For example, when openDemocracy wanted to interview reporters across 47 European Council member states about commercial pressure inside newsrooms, there was little chance for statistical significance.

“Why?” you might ask.

All respondents became whistle-blowers. Whistle-blowers need protection, including not disclosing important real demographic data, such as age or sex. We were expecting some contributions from countries where exercising freedom of speech may lead to severe personal consequences. We decided that providing personal data should not be compulsory; nor, if provided, should these data sit on a server with a company that co-owns our information.

The EU had wildly different and incomplete counts of journalists in the region, meaning establishing a country-level representative sample was tricky.

We couldn’t line up all press unions and associations and randomize respondents because membership lists are private. They also don’t include everyone, although it would have been an acceptable base as long as we were honest about our limitations. Plus, in some countries, transparency projects lead to suppression and we received expert advice in which countries we could not solicit the support of unions without attracting either surveillance or punitive consequences.

In cases like this, you needn’t throw the baby out with the bathwater.

We didn’t.

Instead, we identified what mattered for our reporting and how polling methods could be adjusted to deliver stories.

We decided that our main focus was examples of commercial pressure inside national newsrooms; whether there was a pattern of how they happened; and whether patterns matched across the region. We were also interested in the types of entities accused of image-laundering activities in the press. We went ahead and built a survey, based on background interviews with journalists, media freedom reports and focus group feedback. We included sections for open answers.

We pushed the survey through all vetted journalism organization channels. In essence, we were not randomizing, but we also had no control over who in the press took it. We also had partners—including Reporters sans frontières, the National Union of Journalists and the European Federation of Journalists—who helped spread the questionnaire.

The feedback coming through the survey was added to a unique database, assigning scores to answers and counting respondents per country, drawing comparisons between anecdotal evidence (issues reported sporadically) and systemic issues (problems reported across the board).

The open text fields proved particularly useful: Respondents used them to tip us. We researched their feedback, with an eye for economic censorship patterns and types of alleged wrongdoers. This informed our subsequent reporting on press freedom.4

Although we did publish an overview of the findings, we never released a data breakdown for the simple reason that the selection could not be randomized and country-level sample sizes were not always reached.5 But we built a pretty good understanding of how free the press is according to its own staff, how media corruption happens, how it evolves and sadly, how vulnerable reporters and the truth are.6

So, are there rules for breaking the rules?

Just a few. Always describe your efforts accurately. If you polled three top economic government advisers on a yes–no question, say so. If you interviewed ten bullying victims, describe how you chose them and why them in particular. Do not label interviews as surveys easily.

If you run a statistically significant study, have the courtesy to release its methodology.7 That affords the necessary scrutiny for your audience and experts to trust your reporting. No methodology, no trust.

Don’t be the next biggest “fake news” author. If an editor is pushing you to draw correlations based on inferences rather than precise data collection, use a language that does not suggest causality or scientific strength. Our job is to report the truth, not just facts. Do not use facts to cover up a lack of certainty over what the truth is.

Where does your story lie? In a pattern? In an outlier? Decide what data you need to collect based on this answer. Figure out where and how the data can be obtained before you decide on the most appropriate methods. The method is never the point, the story is.

If you run a survey, field-test your findings and protect your reporting against potentially problematic claims. For example, say a survey suggests that the part of the city you live in has the highest crime rate. Yet you feel safe and experienced almost weekly street violence in another neighbourhood you lived in for a year, so you may not yet trust the data. To check if you can trust your data, visit the places that you compare and contrast; talk to people on the streets, in shops, banks, pubs and schools; look at what data was collected; are residents in one area more likely to file complaints than residents in another area? What types of crime are we talking about?

Have the types of crime considered in the analysis been weighted, or does a theft equal a murder? Such “ground truthing” efforts will allow you to evaluate your data and decide to what extent you can trust the results of further analysis.

Footnotes
1. See 130 definitions of data, information and knowledge in Zins, C. (2007). Conceptual approaches for defining data, information, and knowledge. Journal of the American Society for Information Science and Technology, 58, 479–493.
2. news.trust.org/spotligh...
3. www.raosoft.com/samples..., www.surveymonkey.com/m..., www.surveysystem.com/s...
4. www.opendemocracy.net/...
5. www.opendemocracy.net/...
6. www.opendemocracy.net/...
7. news.trust.org/spotligh...

Written by Christoph Koettl

During the violent clashes in Cairo in August 2013 there was one particular YouTube video that received a lot of media attention. (The original video was subsequently removed from YouTube, but can be also viewed here). The widely used description for this video, which for example appeared in the headline on a Washington Post blog post, was that protesters had pushed a police car off a bridge in Cairo.

Violent behavior displayed by protesters is, of course, relevant when investigating dispro- portionate use of force by the police, as we at Amnesty International do. We also work to verify video as part of determining whether human rights abuses have occurred. As a result, this video represented important footage that needed careful review.

What stood out from this video, in contrast to the description and resulting headline, was that at no time could the protesters be seen actually pushing the car off the bridge. It clearly required a closer look. Here’s what I did to assess the content of the video and determine the exact location of the incident:

One of the first steps when validating citizen video is to search for other content that shows the same incident.a I normally search YouTube as well as in the Storyful dashboard (a paid service) and Storyful’s Open News Room to find additional video content. (As noted in the chapter, I filter my YouTube searches by upload date to narrow down the number of results.) Using these tools, I found a second video that was shot from a different angle. It appears to be filmed from a nearby high-rise, and thus provides a great view of the whole scene. The additional footage shows that no one actually pushed the police car off the bridge. Rather, the car appears to have collided with another vehicle, causing it to roll back and fall off the bridge. This second video confirmed the incident was real, but also revealed that the description (and headline) were inaccurate.

With the new vantage point provided by the second video, it became easier to find the exact location of the incident. The Washington Post article provided the “6th of October Bridge” as the setting of the video. This is sufficient to get started, as the bridge is easy to find on online maps. However, the bridge is actually a very long elevated road that runs through large parts of the city. This made it more challenging to find the exact location.

When carefully reviewing the second video, one landmark stood out: a sports stadium. By tracing the 6th of October Bridge on Google Earth, I was able to identify two stadiums that are in close proximity to the bridge. After rotating the view on Google Earth to find the potential location and line of sight of the person filming, I found a location that matches up with the second stadium. Having confirmed the general location, it was then easy to pinpoint the high-rise buildings overlooking the incident. Using the mapping tool in Google Earth Pro, I produced a simple overview map, depicting the location of the two videos, the area of sights, and relevant landmarks:

Finally, two more features further confirmed the location: A broadcasting tower is visible in the background of the video, which is also visible in satellite images. Additionally, I turned on the Panoramio photo layer in Google Earth to check for user-generated photos. The Panoramio layer contains georeferenced, user-generated photos that provide an on-the-ground view, and thus a high level of detail. There are also several photos from underneath the bridge where the car landed, and the pillars of the bridge as seen in the video match up perfectly.

Thanks to a combination of video searches, Google Earth and Google Maps, I was quickly able to verify where the video was shot, and to also debunk an erroneous description that could have had serious implications for the protesters in Cairo.

Coordinates of lead video: 30.058807, 31.303089

In the end, after the real story of why the police car fell off the bridge was clear, The Washington Post followed up with a second post and a correction.