Epidemiology is a fascinating field and one that takes a lot
of education and experience to do well. Unfortunately, it is also a field rife
with amateurs wandering around in a Dunning-Kruger
haze. Who hasn’t read about Jenny McCarthy with her self-styled degree from the
University
of Google? Using only her Google skills she claimed to be able to tease
globally relevant autism research from her personal experience with her
son. No one can deny Ms. McCarthy’s
personal experiences but from a policy perspective it isn't enough. Anecdotal data can be a real
problem because anecdotal data can be incredibly compelling. It can tug at the
heartstrings and cause otherwise rational individuals and news organizations,
(like say the Toronto
Star) to go places where they best not go.
While the Toronto Star/HPV story is still emerging, it is nowhere near the
worst recent case of good intentions gone wrong in the field of homespun
epidemiology. In my opinion that award has to go to the ongoing Fort Chipewyan
cancer story.
For those of you not familiar with this story, a local
family physician in Fort Chipewyan noticed what he felt was a “cluster” of
unexpected cancers coming through his local practice. Based on this small batch
of anecdotal data he raised an international alarm against oil sands
development in the region (ref).
The authorities, doing as would be expected, brought in a team of professional epidemiologists
who did a comprehensive analysis and eventually established that no such cancer
clusters existed. Now you would think this would represent an open-and-shut
case right? Well you would be wrong. As is typically the case when health risks
are drawn into the political world, the spinners started spinning. The anecdotal
data was described as “an
ongoing tragedy” and was broadcast around the globe almost immediately, because
that is what extravagant claims do. Meanwhile, the comprehensive work of
collecting, compiling and analyzing the data had to be completed. An initial
report was prepared in 2009 and the final report completed in 2014 (presented
in full at the Alberta
Health Cancer page). This report completely vindicated the public health
officials and debunked the built-up claims of the activists (as described in
the CBC
follow-up report :“Higher cancer
rates not found in oil sands community, study shows”). That being said the
original reports from the local doctor were broadcast in 2003 so for the better
part of a decade the Alberta Government and the various oil sands producers in
the area had to deal with public-relations firestorm after public relations
firestorm based on a small sample of anecdotal data and a self-schooled “expert”.
As described, the problem faced in the field is that doing
good science can take a long time. For those of you unfamiliar with what an epidemiological
study entails here is a short summary. Initially health records (some still in
paper form) have to be processed, input into information systems and anonymized
(for ethical and privacy reasons). Now we are not talking about one or two
records here but rather thousands of records and the records need to include
all sorts of accompanying information needed for the analyses. The data must then
be examined and confounding variables (like age, smoking habits etc...) have to
be considered since any detailed statistical analyses have to be able to control
for those confounding variables. What do I mean by that? Well we all know that
smokers are more susceptible to certain types of cancers, so if your population
has a lot of smokers you have to recognize that some cancers will be related to
the smoking and not the oil sands. All this work takes time. In the end the
waiting is typically worth it. In this case, as described by Dr. James Talbot,
Alberta’s Chief Medical officer, the rates of cancers were comparable with the
rest of the Province (ref).
Of particular interest, the cancers reported as being an issue by the local
physician included cervical cancer (which is primarily caused by a virus not
environmental causes) and bile duct cancer (which also has strong
non-environmental risk factors). Any reporter worth his/her salt could have
looked up details of these two cancers and perhaps included those useful details
in their reports, but instead it took years to thoroughly demonstrate that the "oil sands caused cancer
clusters" were no such thing. Ultimately for the oil sands producers and the
public health officials it was a pretty pyrrhic victory. Sure in the end the oil sands were proved to not be at fault, but like the Olympic gold medallist who gets his medal taken away from him a decade later, the gold medal had
long since been used by the activists in their campaigns and the all-important financial
rewards of increased fundraising for activist groups had been achieved.
You would think that a decade’s worth of empirical study
with thousands of hours of work and millions of dollars of effort would buy you
some peace of mind for a little while...but this is the oil sands so that is simply not going to
happen. Mere months after the Alberta Health report came out a new study was
produced, this time by a group of ecologists (One River web
site with reports). The McLachlan report is a tremendous piece of work and for
anyone interested in the field it is well worth the time to download and peruse.
Unfortunately, it does not do what all the activists who cite it or journalists
who wrote about it claim it does. The report is another grand example of the
power of the anecdote. For outsiders this should not have been a surprise after
all, the lead researcher
was quoted as saying:
“I am not a physician. I am originally an ecologist by training. What do
ecologists do? They look for links between organisms and their environments.
What we have done is to link environmental wildlife and human health. Something
that reductionist science would never be able to do.”
and
"The study was holistic in approach. It examined the linkages between
the environment, wildlife, diet and human health. In that way it complements
and resonates with Traditional Knowledge which is similarly holistic ."
I suppose I am a bit old school but whenever I hear a
scientist using the term “reductionist science” in that manner while repeating
the word “holistic” my ears perk up. In this case my concerns were apparently well
deserved. The report represents a fascinating blend of ethnocultural and ecological research but it is a survey report pure and simple. It purports itself to be an
attempt to “characterize the impacts of
upstream industrial activity associated with the Athabasca Oil Sands for
wildlife, environmental and especially human health as it affects the MCFN and
ACFN” but the report suffers from a severe lack of data with so few analytical results that the statistics and conclusions have little or no statistical power. The report provides a terrific snap-shot into community beliefs and
illness records but does so without the benefit of context or addressing confounding
variables. The data is expository in nature with broad chemical conclusions built on a mere handful of
samples (for PAHs they relied on 3 beavers samples, 4 moose, 23 ducks and 8 muskrats).
The report details PAH concentrations from samples collected in areas downgradient of the oil sands
developments but, as I described in a previous post, PAHs are naturally
occurring in the area as a result of the presence of surficial oil sands (some
of which are being mined) as well as yearly forest fire events. From my reading
of the work, no effort was made to obtain background samples. Background
samples would be samples collected from upstream of the oil sands developments,
but within the area of the oil sands themselves. Background samples allow scientists
to establish the natural concentrations of contaminants in the region. It is
impossible to establish whether the samples presented in the report have concentrations
that are significantly higher than the background concentrations if you have never collected a
background sample. Moreover, what little data collected is not presented in
useful form. The authors depend on means with error bars rather than providing
raw data and admit that in the case of the moose PAHs the results were skewed
by a single highly impacted animal. After all this, the concentration of PAHs
that were considered “very high” by the authors were actually half the Health
Canada safe dose. As I described in my previous post, given all the conservatism built into a Health Canada safe dose, if your highest value is half that safe dose then you are doing
pretty well.
So why am I harping on this one study? Well it is the type
of work that generates headlines “Oil
sands pollution linked to higher cancer rates in Fort Chipewyan for first time:
study” (Financial
Post) and can be hyped by activist organizations: “'Alarming' New Study Finds Contaminants in Animals Downstream of
Oilsands” (Desmog
Canada). However, when viewed from a risk assessment and epidemiological
perspective can be discounted as an initial survey that is in dire need of follow-up and a more comprehensive sampling plan. It represents exactly the type
of study that the news and activists love and policy-makers have to fear. The study will drive public perception for years without sufficient data to support that
drive. From a risk communication
perspective, it represents a gold mine for activists, with its headline making
quotations from first nations members, but has no underlying substance to
support those quotations. Most disappointingly, until a carefully designed
study is carried out to either confirm or debunk the results, it will be the last word on the subject whenever anyone
wants to make a headline or solicit funds.
No comments:
Post a Comment