Statistics Gone Wrong – But Where?
It’s the ugly duckling of the scientific world. Very few scientists find it attractive or appealing to begin, however with a bit of time and attention (not to mention a few gallons of coffee) the beauty of it shines through. What am I talking about you ask? STATISTICS of course!
Lets face it, when we signed up for our degree very few of us realised exactly how much time we would spend sitting at a computer. We pictured ourselves in the lab discovering a cure for cancer, saving the world from global warming and (if you’re me) soaking up the sun on the beach while tagging sea turtles! But alas, we have come to the realisation that any data we collect will need to be analysed and presented a results.
But whose responsibility is it to communicate the results to a greater audience in an accurate manner? When a journalist picks up a scientific paper, can we really expect them to understand the statistics? After all they can be incredibly complicated with so much room for error and misinterpretation.
Let’s look at some examples where statistics have been taken out of context from the Australia Science Media Centre (2011) :
|4 in every 1,000 women will die of breast cancer. This new drug will reduce a women’s risk of breast cancer by 25%||This new drug will reduce a women’s risk of breast cancer by 25%||For every 1,000 women that take the drug, 3 (instead of 4) will die of breast cancer. In terms of a women’s overall risk – the drug gives a 0.1% reduction|
In this example we can see that while the conclusion is correct, it is very misleading to the general population – especially if it were to be published without the initial statistic that 4 in 1,000 women will die from breast cancer. If you were to explain that the absolute risk would be reduced by 0.1% as opposed to the relative risk of 25%, I’d hazard a guess that fewer women would be willing to take the drug.
Another classic issue in reporting statistics, whether its a scientist or a journalist is framing of messages. Again, it isn’t incorrect or inaccurate, it just puts a spin on the statistics to make it look better than it is. For example the classic scenario from our friends at Grey’s Anatomy:
Young mother: ‘Doctor, if I have this surgery to remove my brain tumour, what are the chances I’ll survive?’
Surgeon: ‘there is a 50% chance of dying on the operating table.’ Umm, no thanks. However, if the surgeon says, ‘there is a 50% chance of you surviving ‘… well now you’re talking!
I know your logical and scientific brains are really angry at these statements and so they should be. It’s the same statistic. She would have a 50% chance of dying or surviving, but how much better does ‘surviving’ sound?
Goldin (2011) highlights the issues in media reporting about breastfeeding as a solution to the obesity crisis amongst children (don’t worry I didn’t realise there was a crisis in kids this young either). Despite a study clearly concluding that babies were gaining weight due to eating bad foods such as sugar drinks and French fries, the media reported that mothers should be breastfeeding their babies instead of using formula (Giselle will be happy). The fact that they came to this conclusion is staggering as the study did not look at breastfeeding at all! The journalists totally missed quoted the science as they picked up on a sentence in the article which stated there was a correlation between formula use & obesity, however this was related to the lifestyle of the child as opposed to the lack of breast milk.
When/where have you seen some seriously misconstrued statistics or results? Do you think that it is more the responsibility of the scientist to make their findings clear or do the journalists need to work on their statistical understanding?
Australian Science Media Centre (2011) Available from: <http://www.aussmc.org/for-media/tips-on-reporting-science/#Communicating%20statistics%20and%20risk%20responsibly> [19 March 2012].
Goldin, R. (2011) Can breastfeeding halt obesity – or is the media misreading the research? Statistical Assessment Service. Available from: <http://www.stats.org/stories/2011/breastfeeding_halt_obesity_jan21_11.html> [27 April 2012].
Getty (2012) A man stressed at his computer.About.com. Available from: <http://learningdisabilities.about.com/od/parentsandfamilyissues/tp/Stress_Management_Tips.htm> [10 May 2012].
Giselle Bundchen (2011) Available from: <http://www.bio27.com/tag/giselle-bundchen-age> %5B10 May 2012].
The Orangutan Project (2012) VIDA Australian Volunteers for International Development Program. Available from <://www.orangutan.org.au/newsletter-articles/vida-australian-volunteers-for-international-development-program> [10 May 2012].