Finding data outliers for solutions journalism
Conversations with Data: #70
Do you want to receive Conversations with Data? Subscribe
Welcome to our latest Conversations with Data newsletter.
It's no secret that media trust is at an all-time low. Dwindling readership and disillusioned audiences are just some of the symptoms from continual doom and gloom reporting. But one way to engage with audiences is to introduce solutions stories to your news outlet: investigating and explaining, in a critical and clear-eyed way, how people try to solve widely shared problems.
To help better understand this, we caught up with investigative data journalist Matthew Kauffman from Solutions Journalism Network in our latest Conversations with Data podcast. He explains how data journalists can apply a solutions-oriented mindset to their work and how to identify "positive deviants" -- outliers in data that might point to places that are responding to social problems.
What we asked
What led you to become an investigative data journalist?
I was always interested in journalism. In high school, I worked on the school paper where I had written some pieces that got some praise from teachers and fellow students. I had thought about a career in law and politics. However, I found journalism appealing in the sense that you could explore issues and educate people without having to sacrifice your own thoughts and beliefs.
I spent many years on the investigative desk at the Hartford Courant. I often thought of it as the projects desk than strictly investigative -- doing deeper dives into issues. You can't view the world without recognising there are issues that deserve a spotlight. There are inequities out there and having the opportunity to bring those to light was valuable. I don't see myself as an advocate, but it felt meaningful when I wrote about stories where there seemed to be problems -- people took notice and change happened.
As for data, I was always comfortable with numbers and math. A lot of journalists are not. I could use an Excel spreadsheet and suddenly you're the data journalist on staff. I have certainly come to realise the power of data in journalism by providing hard evidence to go along with anecdotal information. It makes for a really strong combination when you have the human element through anecdotes, but then hard systemic evidence provided through data.
How would you define solutions journalism?
The concept of solutions journalism is this idea that journalists can and should apply the same kind of rigorous, clear-eyed examination of responses to problems as we do to the problems themselves. So it's not picking winners. It's not advocacy. But we're very good at digging into systems and seeing what's going on and certainly identifying problems.
And if you have in particular a case where an issue is well known locally, for instance, you've already done 10 stories on how terrible the opiate crisis is in your community, it's important to consider whether another story about how awful things are is necessary. I think there's better value in saying, well, what have other communities done? Are there other places that have bent the curve on this? And if so, are there specific policies that they put in place to kind of move the needle? Have they found a successful response? So that's the concept of solutions journalism.
It's not a replacement for accountability journalism or reporting on problems. But I think it is a really strong partner in going beyond simply reporting on the problems and showing how success in other communities might be replicable. We call it 'hope with teeth'. Readers and viewers want journalism that sort of goes beyond doom and gloom. Studies do show readers respond to stories that provide a possible path forward.
How does data tie into solutions journalism?
Data are a terrific partner with solutions reporting. When the impact of a response can be measured with numbers, data are a terrific path to that. It's not enough for a town official to claim a terrific response on covid or homelessness or the achievement gap in schools without showing evidence. For instance, here's the response that was put in place. Now ask what difference has it made.
I use an example of hospitals in California that put policies in place to try to improve maternal health during pregnancy, essentially avoiding mortality specifically to address postpartum bleeding. Ok, that policy sounds like a good idea and seems like it will work, but do we have evidence that it works? And the data provide that evidence: while mortality was going up in most of the country (at the point at which these provisions went in place in hospitals), in California the curve was bent for maternal mortality. For instance, you can see in the data that suddenly the line stops going up and starts going down.
Of course, there are always issues with causation and correlation. Just because something happens after something else doesn't mean the first thing caused it. But that seemed pretty solid evidence that the changes that were made led to this better outcome. And these changes could be made at hospitals in any state.
Is solutions journalism only for more seasoned journalists?
No, I think one of the nice things about it is that there isn't a steep learning curve. It's just having a mentality about how you approach stories. And if you approach stories saying, I want to fully understand this issue: who it's affected, how bad it is, what caused it? Those questions take you to the past, up to the present, where and how did we get here and where are we? And it's just adding that additional part -- where could we go from here? Problems are rarely isolated to one single institution, one single school, one single town or one single country. If you expand your horizons beyond how did we get here and where we are and think about the places that have found a path forward -- that's all it takes. Any journalist at any news outlet of any size anywhere can adopt and pursue solutions journalism.
What are the biggest stumbling blocks journalists encounter who are new to solutions journalism?
I think the biggest stumbling block or hesitancy is this concern about, 'gee, this sounds a little like advocacy to me'. The first I ever heard of solutions journalism, I happened to be at a conference in Boston and there was a panel I sat in on with someone from the Solutions Journalism Network talking about this. And I went to them afterwards and I said, 'This sounds really intriguing, but how is this not advocacy?' And so I think a lot of people need to get comfortable with that first step. Remember, it's not about here's the right way. It's not about picking winners. It's just that there are other places that have approached the same problems that I'm writing about. So here's what they did. And then, it involves getting your investigative cap on, as all journalists have, and really digging in. Did it make a difference? Can you tie what they did to this better response? And then the key thing is making sure and being comfortable that this isn't public relations, it's not advocacy. It's as simple as having a mindset to produce better journalism.
How does reporting on the limitations of a particular solution come into play?
Reporting the limitations is a critical, essential part of solutions reporting. It's not a solutions journalism story if you don't explore what the limitations of a response are. For instance 'Hey, this worked in this community, but it's also really, really expensive or it worked in this community. But this community has great public transportation and that was essential to the response. And it won't work in a community that doesn't have that or it works for this demographic. But they really haven't solved the problem for this demographic.'
How do you do a sanity check when using data in your solutions reporting?
The first step is to think about what are the questions that I have that these data can answer. And if you're looking for a positive response or "success", it is important to ask what does "success" mean in this case? And it doesn't necessarily mean, whatever place has the highest number because they're all sorts of reasons why one community, school, hospital or institution might have a higher performance metric than another. So maybe you are looking at the improvement over time. For instance, is my measure of success equity across different groups? So the first thing to do is interview yourself to ask what am I looking for. What to me would be evidence of a successful response.
Interviewing your data is absolutely critical. Ask yourself if you can trust the data. Where did these data come from? Do they seem to be reliable? If I'm comparing multiple places, am I sure that these data are comparable, that they were captured in a similar way? And then when you do the data analysis, look for a certain smell test. For instance, if something went up 17,000% -- that doesn't sound right. Did I do the math wrong or is there missing data for certain years? If so, then I can't work with this. Even before you analyse the data, doing that interview and saying, does it appear the data are clean is critical. Remember that all of the caveats that we have in all data journalism certainly apply to solutions journalism. It's simply about getting a lead to a place to investigate further and see if they belong in your story.
Finally, what kind of training does the Solutions Journalism Network provide?
We have training in 17 different languages. We have journalists and regional managers in Africa, Europe and the United States. We have a ton of webinars and training. You can visit Solutionsjournalism.org, Learning Lab or the Hub. We do live web-based trainings. It's best to start with our Solutions Journalism 101 course. Those are held live once a month. If you don't want to do a live webinar, you can find asynchronous training on our site, too. We then have a more advanced training that gets into some more details about solutions journalism and the different ways to frame a story looking at different slices of a story. That lends itself to a solutions resource I'm currently developing. I also recommend checking out the Solutions Story Tracker with more than 1,000 stories on how people are containing COVID-19, coping, and caring for each other during the pandemic.
Latest from DataJournalism.com
Sonification is all around us! So how can you use music and sound to enhance your storytelling? Data journalists Duncan Geere and Miriam Quick explain how by walking us through some compelling examples from journalism, science and civil society. Read the entire long read article here.
Our next conversation
Our next Conversations with Data podcast will feature economist and data visualisation expert Jonathan Schwabish. He is the founder of the data visualisation and presentation skills firm, PolicyViz, and a Senior Fellow at the Urban Institute, a nonprofit research institution in Washington, DC. In addition to his research at the Income and Benefits Policy Center, he is a member of the Institute’s Communication team where he specialises in data visualisation and presentation design. He will speak to us about his latest book Better Data Visualizations: A guide for Scholars, Researchers, and Wonks.
Tara from the EJC Data team,
P.S. Are you interested in supporting this newsletter as well? Get in touch to discuss sponsorship opportunities.