skip to Main Content
This is a photo of a person cutting apart pages of printed text.

Keep It or Cut It?
Thoughts on Simplification

In this blog post, I talked about researchers’ fear of leaving information out of their research communication products.

This week, I’m digging a little deeper into the specifics of what types of information (in my opinion) are and aren’t OK to leave out.

Scientists, on the whole, deal in nuance and caveats:

Variable A is associated with Variable B. One does not necessarily cause the other, and we only know that they are associated under a particular set of conditions, in a particular sample, with an acceptable-but-non-zero margin of error.

Unfortunately, nuance and caveats can reduce the impact of science communication. Generally, short, simple, black-and-white messages grab more attention, and they’re easier to understand and remember:

Variable A causes Variable B, always, everywhere, and forever.

So, where do you draw the line between overly simplistic sound bites and a quagmire of details? For me, the litmus test is about the message that the audience will take away and the actions they might, or might not, take based on that message.

Example 1:

You’re writing up the procedures you used to conduct a research study, as a prelude to your write-up of the study results.

Problematic simplification: The audience for your communication product will be evaluating the quality of your procedures (e.g., as in a peer-reviewed journal article) or will be responsible for replicating your procedures (e.g., as in a technical manual). In these cases, I think any simplification could lead to misunderstandings or mistakes; I’d err on the side of including details.*

Acceptable simplification: The audience for your communication product will be anyone other than the folks I mentioned above. School principals don’t need to know whether you randomized assignment to experimental conditions at the school level or the district level. Policymakers don’t need to know which statistical analysis package you used. Most audiences will care about, and will make decisions based on, your results. For these groups, I think it’s OK to describe your procedures in pretty broad strokes…you surveyed 1,000 students nationwide, or you looked at disciplinary records from New York City public schools for the past 5 years…just enough information to give some context.

Example 2:

You’re developing a fact sheet about your research study on teen vaping.

Problematic simplification: Your research sample only included girls. Your fact sheet talks about “teens” and never mentions that your sample did not include any boys. Parents, educators, and public health officials could unknowingly make decisions that are completely wrong for half the population.

Acceptable simplification: Your research sample of 1,000 teens excluded 7 teens who were absent from school on all of the testing days. I think it’s OK to not describe this exclusion in a brief fact sheet because 0.7% missing data will have no meaningful impact on the results.

Example 3 (for the more stats-y folks in the crowd) :

You’re creating an infographic about test scores for girls vs. boys. On average, girls scored higher than boys on a particular test.

Problematic simplification: When you statistically adjust for a bunch of other variables (e.g., age, race/ethnicity, parent education), the difference between girls’ and boys’ scores disappears. In this case, presenting the unadjusted/raw mean scores (i.e., without controls) for boys and girls would be misleading, suggesting a gender-based difference when differences are, in fact, driven by other factors.

Acceptable simplification: When you statistically adjust for a bunch of other variables, the difference in girls’ and boys’ scores gets a little bit smaller, but the difference is still statistically and practically significant. I think it’s OK to communicate just the unadjusted means because the overall conclusion drawn from unadjusted vs. adjusted means would the same, and unadjusted means are easier to understand.

So, like everything else in research communication, I think decisions about details to include and exclude should be driven by audience and purpose. Give your audience whatever information they need to make well-informed decisions; anything else is clutter that jeopardizes attention, understanding, and action.

Need help deciding what to keep or cut from your latest research communication product? Data Soapbox can help!

* If you’re bumping up against a word count limit, consider referring readers to other detailed documents.

Image source: https://www.pexels.com/photo/cutting-of-papers-7128657/

This Post Has 0 Comments

Leave a Reply

Your email address will not be published.

Back To Top