Assessing Science Amid the Pandemic

Science is making a comeback. Putting aside the lazy, if not insulting cliché of scientist as nerd, as if studious curiosity was a character flaw, the world now looks to science professionals with gratitude and hope. Now is a good time to move past careworn excuses, like “I’ve never been good at science/math/technology,” and recall the science portion of our basic arts and sciences education. Here are some tips to hone atrophied critical thinking skills as they apply to scientific data presentation.

Many of us are anxiously checking graphs of the number of Covid-19 cases, looking for that hopeful “flattening of the curve” (or more recently, lowering from the plateau). How does the graph scale effect the “shape” of a curve, and our resulting perception? The two graphs below show the same data, a doubling of cases per day over 15 days.

The data on the left graph are plotted on a “linear” scale, where the distance between each unit on both the horizontal (x) and vertical (y) axes increases at equal intervals. The right graph is called “semi-logarithmic” because intervals along the y-axis increase by a factor of 10, enabling the viewer to see detail of the wide range of number of cases. The shape of the curve, then, is strongly influenced by the choice of the axis scale.

Scale is important, and can effect bias. Generators of graphs, in the same way as news spinners on cable TV, often draw from a grab bag of data illustration tricks to bias the representation. One trick is using an abbreviated scale, such as choosing to “omit the baseline.” This example shows the percent of polled individuals who stated a party preference and agreed with a particular court decision (the Terri Schiavo case).1

The choice of the abbreviated scale on the left was chosen by the cable TV channel to exaggerate the political differences between the polled individuals, rather than illustrate their similarities as shown on the complete dataset on the right.

The difficulty of collecting and modeling accurate polling data was demonstrated spectacularly after the 2016 election. Political models, like all future-casting, are only as accurate as the input data and the assumptions underlying the model. People are notoriously bad self-reporters. Pollsters usually ask the same question several times in different ways in order to try lower the error rate of the responses. Note: this is a fun game to play with friends when the pubs open up again.

Most scientists, however, are not fortune tellers and are uncomfortable in this role. When accurately reported, scientists will intersperse their comments with nonglamourous terms like “error rate” and “statistical significance” to try and convey the limitations of the conclusions that can be drawn. Science deals with populations, not individuals. This can be frustrating for an individual who, for example, wants to know if they should cancel their summer camping trip this year.

Many readers will scan graphs in popular media and draw the most obvious conclusion. It is difficult to convey uncertainty and variability of the data behind the graph in a concise, easily digested illustration. Also, it is our natural tendency to look for
significant meaning behind apparently correlated events. Let’s say that, back in the good old days, you go to a restaurant, eat the daily special with some kind of suspicious orange sauce, and the next day you have a headache. You might draw the unfair and
potentially biased conclusion that the restaurant served you bad food. However, the scientific method would suggest that you consider all of the possible variables, including the two cocktails you had prior to dinner. More data collection is required

A classic example used by data scientists to illustrate the adage “correlation does not imply causation” was published by the Church of the Flying Spaghetti Monster. (2) Church members, known as Pastafarians, believe pirates are divine beings, and that global warming, earthquakes, hurricanes, and other natural disasters are a direct effect of the shrinking numbers of pirates since the 1800s. The intended tongue-in-cheek conclusion is that more pirates should solve the problem of climate change. The actual
take-home message is that just because two things are correlated, there may be no cause and effect between them.

After considering the source (and ridiculousness) of the data presenter, however, a sharp-eyed, critical viewer may notice other problems with the illustration. What kind of scale is used on the Pirates axis? The scale is backwards, non-linear, and not even in
order. So, not only is the correlation suspect, the visualization is poor. A better representation of the data might show the temperature increase by year, and then the pirates counted during each pirate survey scaled to the size of the total:

Now the data, if accurate–another important question–are even more interesting. There was a big drop in the number of pirates between 1920 (15,000) and 1940 (5000). What happened, was there anti-pirate legislation passed? Were there really only 17 pirates in 2000? How do they count pirates, anyway? These are the questions of a curious person.

So, in addition to thinking about data source, accuracy, correlation and scale, another tool in the critical evaluation toolbox is context. Here is a graph, on a linear scale, showing the rate of new deaths per million people each week for the United States as of the end of April.( 3 ) The accompanying article provides the data sources, how the data were processed, assumptions made, and caveats, providing confidence in the contextual information:

What each individual will draw from this information depends on their personal history and perspective. The dramatic rate of increase of the virus is alarming, of course. But imagine if the public demanded government action on cures for cancer with the same urgency as for Covid… once we flatten that Covid curve back down to the level of car crashes.

Perhaps this laser focus on medicine and science will result in a blossoming of interest among young people as happened during the space race in the 1960s. As a young girl, I wrote to NASA about becoming an astronaut and was told that only men were eligible, but there were plenty of other ways to get involved with the space program (#crusheddreams).4 Now the doors are wide open for young people to major in the STEM (science, technology, engineering, math) fields. Their parents, however, are sending signals that science might be “too hard.”5

Really? Too hard? What happened to the generation that did things “not because they are easy, but because they are hard?” (6). As we imagine our post-Covid world, perhaps this perception will change as well. We all know now that the work of science is critical and, more importantly, personal. We are all scientists now.

---

Peggy Myre is founder and president of Exa Data & Mapping, a local company providing marine environmental data management and visualization services to national and international clients.