20211223 - assessing claims

in the context of the pandemic, misinformation and theories

I was listening to Dr Karl on the radio one day and a caller asked him why he loves science so much, and I really liked his response which was something along the lines of Science is like a tool that can help you to not be fooled.

In that theme, I'm writing about some of the (mental) tools I'm aware of that I've found helpful to avoid being fooled by some of the arguments or claims I've heard or read about regarding the covid-19 pandemic.

While the scientific method is a useful tool to protect yourself from being fooled when you're trying to figure out how something works, the "tools" I'll talk about are useful to protect yourself from being fooled by certain lines of arguments or reasoning or "evidence".

There is already a conspiracy theory handbook that talks about a lot of the ways you can tell whether something is worth believing or not, and maybe I'll end up repeating a few of those, but one of the biggest downfalls I could think about the handbook was that one of the first tools they discuss (on how to protect yourself from wrongly believing in a conspiracy theory) is to be "innoculated" or to have the theory "prebunked", which is to say, before you are even exposed to a conspiracy theory, you had a trusted friend already introduce the theory to you as being a conspiracy theory and all the things it gets wrong.

Unfortunately that tool doesn't really help you if you are exposed to the messaging or memes of a conspiracy theory before you have a trusted friend introduce it to you for what it is, a crazy claim with not much solid evidence or rational reasoning, and which if believed, could possibly do you some very real harm.

Maybe it's because a lot of us in the developed world live in some very safe societies, that the holding of incorrect beliefs can be relatively benign, but one need only scroll through a couple Herman Cain award posts to see where incorrect beliefs regarding the pandemic can get you.

So "innoculation" or "prebunking" against an idea shouldn't be depended on because it requires someone near you being switched on and aware of every new conspiracy, and to introduce the details of the conspiracy and why it's false, for it to work.

Wouldn't it be much better to develop in yourself the ability to determine which belief of two opposing beliefs is most likely to be correct?

on to the tools:

logical fallacies

So many conspiracy claims appear on the surface to be using logic to make their argument, but so many that I hear employ logical fallacies.

eg. absence of evidence is evidence of absence

eg. "the news (or the "mainstream media") says theres a pandemic, but no one I know has it or has died from it. Therefore, because I don't have any evidence to prove theres a pandemic, that in itself is proof that there is no pandemic."

... hopefully its clear to see the mistake in this line of reasoning, it could just be a matter of statistics or early days yet as to why no one you know yet has had it.

philosophical razors

eg. occams razor,

If presented with two opposing theories, the one that makes the fewer assumptions is probably the correct one.

It doesn't mean it's true in all cases (hence the "probably"), but it can be a helpful shortcut to save you time chasing down and debunking every single claim that you hear.

a case study:

eg, in December 2021, one of the claims I've seen floating around is that the Queensland government state a figure for doses of vaccination administered to their population, as well as a figure of how much percentage of their population is fully vaccinated, and also that it takes two doses to be considered fully vaccinated, the figures don't line up, therefore, the government's lying to us and manipulating us.

Here's a screenshot of the claim:

The claim was circulating a few weeks prior to when I captured screenshots from the official websites so the figures are only slightly different, for example the claim mentions a figure of 83% but the screenshots below were taken at the time when the figure had since increased to 86%

figure showing the percentage fully vaccinated:


taken from https://www.qld.gov.au/health/conditions/health-alerts/coronavirus-covid-19/current-status/statistics


figure showing how many doses have been administered:

taken from https://www.qld.gov.au/health/conditions/health-alerts/coronavirus-covid-19/current-status/statistics


population of queensland:

Total doses administered is in the 3.2 million range, if it requires 2 doses to be considered fully vaccinated then we could do a crude calculation by just dividing it by 2, 3.2 million divided by 2 is 1.6 million, therefore 1.6 million people should be fully vaccinated, but then the eligible population is roughly 4.3 million, and 1.6 million divided by 4.3 million is only 0.37 or 37% of the population, not 86.6% as the figure claims.

application of the razor:

non-conspiracy-theorists theory could be described as there is no conspiracy, if there is a discrepancy in the numbers it could just be that there's a typo in the numbers, alternatively it could be that one of the figures is using a different type of accounting than what was used to attain the other figure, and thats why they don't line up.

conspiracy-theorists theory could be described as, there is a conspiracy, and this discrepancy in the numbers is evidence of the conspiracy and that means the government is manipulating us, which means not only are they lying about how many are fully vaxxed, they're also lying about the severity of the disease, the contagiousness of the disease, the efficacy of the vax, the fact that there's even a pandemic at all, etc etc.

you can see that the first theory can explain the discrepancy in the numbers with a single assumption, which is that a typo would explain it. Or alternatively that it's just a different set of accounting was applied in arriving at each number.

But for the conspiracy theory to be true, you need to make a lot more assumptions, eg. taking the most extreme theories, they assume that the doctors are in on it. The WHO is in on it. China is in on it. USA is in on it. Our own government are in on it. The PCR tests aren't as accurate as they claim. Vaccinations aren't as accurate as they claim. We don't actually know as much as we are being told we know about how our bodies work. etc etc etc. This is what most people mean when they say that a conspiracy theorist friend has "gone down the rabbit hole", because once you have no problem with the amount of assumptions you need to make to hold a belief, no amount of evidence or test results will save you, because you can always maintain your existing beliefs by simply applying a new assumption, that that particular test result or piece of evidence is fabricated. That the people behind the test are "in on it"... and so on.

Anyway, we don't actually have to assume anything for the above claim, it turned out the answer was just a matter of accounting, like the first simple assumption assumed.

To be fair, this was pretty obvious if you actually read the published figures on Queensland health's website where at the bottom of the first table of data there was an asterisked note inviting you to refer to the data caveats

repeated here (relevant parts highlighted):

Vaccination data: This data reflects the COVID-19 vaccine doses administered by Queensland Health. Some clinics are set up on a temporary basis for the purpose of outreach to specific groups. Additional COVID-19 vaccinations have been administered by the Australian Government in residential aged care and disability care facilities. Data validation exercises may result in retrospective changes to vaccination administration daily and cumulative totals.

When you look at the federal governments figures, you can see that the figure for total administered doses in Queensland was as you would expect (roughly 8 million),

it was only when you see the figures for Queensland broken down by who they were administered by, and see the figure for "administered [by the state]" that you get the same figure as was published on the queensland state governments website (a little over 3 million) (which makes sense) which was the figure that the claim used but mistakenly interpreted it to be the full figure for total doses in the state.

It seems that by not looking more closely into the data and straight away jumping to making social media posts citing those published statistics as evidence for their claims, they had suffered from confirmation bias, as well as a failure to properly interpret published data or results correctly.

Interpretation of results

Often I've found there's been a mistake in the interpretation of results or data or studies, and this has then led to an incorrect conclusion. Three examples I've seen recently:

  1. The previously mentioned QLD doses statistics case study
  2. Belief that a published notice from the American CDC that they were invalidating a particular PCR test was an acknowledgement that the PCR method was flawed and unreliable, but all it was is that they revoked a particular test because it was superseded by a better one
  3. Craig Kelly and Alan Jones on UK statistics of hospitalization and death of vaccinated versus unvaccinated people. (theres a media watch episode debunking this)

Cognitive biases

eg. Confirmation bias: only searching for information or evidence that supports your existing belief and dismissing other evidence that discredits your belief.

Peer review

Sometimes outlandish claims are given a level of authority by citing some scientific studies that they claim prove their argument, but often what they neglect or rather what they often dismiss as being a part of the conspiracy itself, is the concept of peer review in the scientific community, and often the studies or papers that they cite do not have a very good peer review result.

Critical thinking

Critical thinking shouldn't be interpreted as being critical of beliefs that don't align with your own. It means criticise the way you yourself think. Criticise the way you've arrived at your own beliefs. You should test your own beliefs wherever possible, otherwise how can we ever change what we believe in, the confirmation bias is a big one because it results in a problem where people don't change their beliefs even if they're misguided.



back