More on that "Toxic Benzene Plume"

Today’s blog post is intended to provide some further commentary on the “toxic benzene plume” from my previous blog post: Questions about the City of Vancouver May 27th Trans-Mountain Expansion Proposal Summary of Evidence. As readers of my blog know, the Trans Mountain Pipeline Expansion Proposal (TMEP) Summary of Evidence (SoE) presented to the Vancouver City Council on 27 May 2015 (ref) included the results of a modelling exercise which suggested that: 

 

a major oil spill from Kinder Morgan's Trans Mountain pipeline expansion project would expose up to 1 million Metro Vancouver residents to unsafe levels of toxic vapours, and as many as 31,000 could suffer "irreversible or other serious health effects,"(ref).

Needless to say this conclusion garnered a lot of headlines. I saw stories from the Georgia Strait to the Globe and Mail. The problem is that, as I described in my last post, this conclusion fails the smell test. The modelling exercise incorrectly compared the toxicological characteristics for benzene to a “pseudo-component surrogate” that was made up of a mixture in which benzene was a very minor constituent. This resulted in a wildly overstated risk to the public which, I will admit, made for some pretty nifty headlines. This post is intended as a follow-up to my previous post to explain the “surrogate thing” as well as to relate some surprising additional information I have uncovered since my last post.

As discussed, the biggest question from my last post was “what is the deal with the surrogate”? Well the chemical definition of a surrogate is:

a pure compound different from, but similar enough to, the analyte that, when added at a known concentration to the sample prior to processing, provides a measure of the overall efficiency of the method (recovery). Surrogates have chemical characteristics that are similar to that of the analyte and must provide an analytical response that is distinct from that of the analyte (ref).

While the Levelton Consultants Ltd  report (the Levelton report served as the basis for that portion of the SoE) uses the term “surrogate” in a chemical context they did not use the term under its chemical definition. Rather they used the non-technical definition of the term: “one that takes the place of another; a substitute” (ref). As I pointed out previously, modelling is hard and to simplify the modelling Levelton took the theoretical oil in the spill and broke it into 15 “pseudo-components” each of which was then assigned a surrogate for use in subsequent toxicological calculations. One “cut” of the dilbit was assigned the surrogate “benzene”. As I described previously, this resulted in badly skewed risk results because benzene is by far the most toxic component in the “cut” of dilbit for which it was used as a surrogate and appears in the dilbit in much lower concentrations than used in the model. As an analogy, imagine you were tasked with compiling a survey of the animal population of Vancouver. To simplify the survey you didn’t ask your surveyors to try to identify the dogs by species instead asking them to group the dogs by size. For a subsequent risk analysis you then assigned the pit bull as a “surrogate” to describe the behaviour of all dogs smaller than 2 feet tall identified in your survey. Would you then feel comfortable with the outcome of that risk analysis knowing that the analysis treated every Chihuahua it counted as if it were a pit bull for risk purposes? If someone subsequently warned you to stay off the street for fear of being attacked by "surrogate pit bulls", based on this analysis, would you stay off the street? Well that is what they did in this report with benzene.

As a follow-up to my last post I also did a bit of digging into the documents referred to in the Levelton Report. Specifically, I located the Intrinsik and Tetra Tech EBA reports (caution both are large files that take a while to download) used to rationalize the use and choice of surrogates in the modelling exercise. The TetraTech EBA report does indeed use “pseudo-components” as surrogates; however, in their analysis “benzene” is used as a “surrogate” only for the benzene component in an oil spill (confusingly it is thus used as a surrogate for itself only). As such instead of representing around 1% - 2% of the total spill mass (best guess on the number used by Levelton) it was determined to represent 0.088% of the spill mass (a fraction based on the Tetra Tech EBA analysis of the future pipeline composition). In the Intrinsik report, “benzene” is also restricted to the actual benzene component of the dilbit and for toxicological calculations only benzene is compared against the acute inhalation exposure limits for benzene. So when the Levelton report claims to follow an approach that “is consistent with the approach taken with the Human Health Risk Assessment (HHRA)” that consistency does not extend to how they approached the critical component described in the SoE and the one that garnered all the headlines: benzene.

The biggest surprise in my follow-up reading was to discover that this was not the first modelling exercise to examine benzene vapour concentrations derived from a theoretical oil spill in Vancouver Harbour. As I described in my previous post, what made my chemist’s antenna go haywire was the model output which said that in the case of a spill thousands of City of Vancouver residents would be exposed to benzene concentrations over 800 parts per million (ppm). As I pointed out, previous in situ studies (actual studies in the field) of oil spills of comparable API gravity crudes had measured benzene concentration ranging from 7 ppm to less than the detection limit (ref). A study of a lighter crude (with more volatile components than dilbit) (ref) and sampled from a mere 2.5 cm above the oil surface measured instantaneous benzene concentrations ranging from 80.4 ppm to 3.5 ppm. Finally, most everyone in the modelling community must have heard of the modelling study of the Exxon Valdez spill (ref). It, too, got a very similar result to the in situ experiments. Thus, when I read the Levelton report I was more than a bit surprised by the output from their model. Well imagine my surprise to discover that the Tetra Tech EBA report, used as a reference by the Levelton authors, actually included a modelling exercise almost identical to the one carried out by Levelton. The difference was that the Tetra Tech EBA modelling used benzene proportionate to its concentration in dilbit. Unsurprisingly, the resulting outputs were entirely consistent with the academic literature. The maximum 1-hour average ground concentration for benzene was less than 100 ppm over the small portion of Vancouver affected by the spill. Certainly not headline worthy, I will admit, but entirely consistent with the rest of the science out there. Nowhere in the Levelton report, which otherwise references the Tetra Tech EBA report, do they contrast their results to those generated by Tetra Tech EBA. It is almost as if they didn’t want anyone to know that the previous modelling exercise had been carried out and had generated such non-threatening (boring? non-headline worthy?) results.

In the academic community there is a simple rule: if a new study runs contrary to a body of research then it is incumbent on the authors of the study to explain the discrepancy. Sometimes the new study is a paradigm changer, but most of the time it represents an outlier of dubious use in decision-making. Unfortunately, the Levelton report does not explain why its results differ so dramatically from the scientific consensus. More troublingly, it does not even acknowledge the existence of the body of research out there, including an almost identical modelling study, that came to such startlingly differing conclusions. I’m sure the Vancouver City Council, and the local media, would be as interested as myself in finding out why such an outlier result was trumpeted on May 27^th?