Field Guide: Spotting Bad Science in the Wild
Good science, bad science - what’s the difference? One major difference that we talked about in the last article is how research is conducted. Did the researchers interpret their results correctly? Was the experiment done above-board? Do other experts agree with the results or do they raise some serious doubts? The other major difference that we’ll discuss today is whether science is being shared and presented in good faith. Someone with a silver tongue can turn even the best research into “bad science.” Let’s talk about why and how.
First, why would anyone want to spread bad science? Oh, sweet friend, the reasons are many. Maybe a company is very motivated to make you buy what they’re selling. They might twist or turn some data, or leave out any details that don’t look so good if that’s what it takes for you to open your wallet. Maybe a website just wants you to click on their headline. Sensational headlines are so much more tempting than ones that carefully and responsibly summarize the truth, and they drive much more traffic to websites who need clicks to make cash. Maybe Big Corn could make lots and lots of money if we used more corn-based ethanol in our cars. Then they would have a reason to lobby politicians and voters with slick campaigns and dubious claims about how “clean” ethanol fuel is. They just might even forget to mention that increased corn production could lead to catastrophic loss of sea life in the Gulf of Mexico (but nitrogen runoff and algal blooms are topics we’ll talk about another day). Maybe….
The list goes on, but these are some of the biggest reasons why people create and spread bad science. Once it’s on the internet or social media, it’s much easier to react and share than it is to respond thoughtfully and verify the source. While I encourage everyone to take 30 seconds to quickly search the topic or publisher to see if they’re trustworthy, I know that won’t always happen. The next best thing is to stay where you are (on Reddit, Facebook, etc.) and look for these tell-tale signs that what you’re reading is bad science. If it passes the test, or it comes from a great source like a science journal, you can go ahead and hit share with a reasonable amount of confidence. Without further ado, here is the scout’s field guide to spotting bad science in the wild:
False Appeal to Authority: This is a trick as old as time, and is a favorite of people who want to say that climate change isn’t happening. It all boils down to someone saying “listen to me because I am smart and impressive.” I have great respect for my dentist, but I would never let him perform heart surgery on me; the same goes for science. Every person of authority has a particular field of study that they are qualified in, and when they step outside of that field they are no longer an expert. If you see an “expert” at anything besides climatology making claims about climate, it’s a very bad sign. Numerous surveys of climate scientists show that 97-99% of them agree that global climate change is already happening. Further, the overwhelming majority of climate scientists agree that the change is anthropogenic (meaning caused by humans). Notice that I did appeal to authority, but only to the appropriate authority? The next time you see someone arguing against climate change on the news, pay special attention to their title. I’m willing to bet that they will be a scientist or doctor of some sort, but not the sort who is actually qualified in the subject.
What to do when you spot it in the wild: Proceed with caution. These critters may strike out in defense, so only engage them when you’re prepared to stand your ground. Do not try to argue with the points they make as this is often a trap. Instead, question why they are qualified to give an opinion on the topic. Asking what specific climate training they have will often force them to retreat to their nest, which is just what you want.
False Correlation and Causation: Just because two events always happen together, that does not mean that one causes the other. The number of shark attacks rises when ice cream sales rise. Ice cream is not causing sharks to attack people; those two events are both caused by a third invisible factor - hotter summer days that make people want to cool off with ice cream and saltwater. It is very easy to assume that when events occur together, they have a cause and effect relationship. You must resist this urge at all costs! This is a pitfall of many bad science reports.
What to do when you spot it in the wild: Exterminate. These weeds creep up pretty often and can do some damage when you let them grow. Luckily, they can be kept at bay by simply calling attention to them and pulling them out. Teach this skill to your kiddos - you don’t have to know much about a topic to know that correlation does not equal a causal relationship.
Cherry Picking Data: When something seems too good to be true, it usually is. If someone only wants to tell you the positive aspects of an idea, you should be suspicious. This world is complicated and even the best products and ideas have a few cons. The positives may very well outweigh the negatives, but don’t trust someone who won’t even admit to any negatives. This usually means that they are purposely hiding a big problem, or someone has hidden the problem from them.
What to do when you spot it in the wild: Only engage when you are carrying the right equipment. A quick search on the internet of “xtopic’s negative aspects” should show you whether you source is trying to hide something, which probably means it’s not a source you want to share. It’s ok to share positive news, but it’s not ok to hide the parts of the truth that you don’t like. If the pros really outweigh the cons, then no one would need to hide the cons in the first place. If you’re not in a position where you can look things up on the spot, use your best judgement. Remember, if it looks like scat and it smells like scat....
Sensational or Charged Headlines: When was the last time a tabloid won an award for journalism? Headlines that are sensationalized are almost always grabs for attention and don’t bode well for the veracity of what follows. In a similar vein, politically or socially charged language is used to fire people up rather than educate them. If the article is intended to get an emotional response out of you, it is probably not giving a fair or well-rounded view of the topic.
What to do when you spot it in the wild: Train yourself to recognize the markings of poisonous specimens so you can avoid them: words like “unbelievable,” “incredible,” and “amazing new ___” should generally be avoided. Also avoid articles that are politically charged with words like “heartless conservatives” or “liberal sheep.” These types of articles come from both sides of the aisle and have a clear agenda to manipulate you. You’re smarter than that, and good scouts never share unfair propaganda.
Hot tip: posts that reference “global warming” instead of “climate change” are typically there to sow disbelief or are simply outdated. These days climatologists use the term “climate change” to reference the earth’s changing climate because it includes more effects than just the rising temperature.
Over Simplifying: Have you ever watched a toddler use all the right puzzle pieces but put together a brand-new nonsensical puzzle? Some people are skilled at doing this with facts. They start with things that are all true, but shave off the details so they can steer the “facts” to an incorrect solution. Let’s look at an example: “The planet has been warming up and cooling back down in cycles for billions of years. That’s what is happening now and it’s to be expected.” See how this statement starts with a fact and ends with a bogus conclusion? (If you’re not sure why this statement is false, read this article next.) This is a powerful tool for spreading bad science, and it is also one of the hardest to spot.
What to do if you spot it in the wild: Leave these nasty webs to people who are trained to handle them. When these are artfully crafted, they can be hard to spot in the first place and even harder to pick apart. Your best defense is to show the argument to a climatologist or to do some research. If you want to earn your detective badge, try researching some of the main facts in the argument and seeing what conclusion the scientists usually draw. If the climate scientists have a different conclusion than what is stated in the web, always trust the scientists.
