When peer-review is not enough - On estimates of avian deaths attributable to coal and nuclear facilities

This afternoon I was directed (via a tweet from Dr. Judith Curry) to a recent article in the New Yorker by Jonathan Franzen, titled: “Carbon Capture: Has climate change made it harder for people to care about conservation?” The article addresses a topic that is close to my heart and about which I have written a lot in this blog. Specifically, it addresses the threat to ecological communities (in this case birds) posed by the international fight against climate change. As I have discussed in my posts on biofuels and other renewables, I see renewable energy alternatives as a vital part of the fight against climate change; but I also recognize that the implementation of many of these technologies poses significant risks to ecological communities. The Franzen article elicited a strong response from Joe Romm at Climate Progress. The centerpiece of Joe Romm’s “take-down” was a graph from a US News & World Report (USN) article Pecking Order: Energy’s Toll on Birds. The graph in the USN article purports to show the relative avian mortality derived from various energy sources. Two of the most contentious numbers in the graph come from a paper titled “The Avian Benefits of Wind Energy: A 2009 update” by Benjamin Sovacool and published in Renewable Energy 49(2013)19-24 (Ref). Those of you who have followed my blog know I have issues with the Sovacool paper and I remain amazed at how far its influence has spread in the fields of renewable energy and climate change. This paper has come up in my discussions of nuclear energy, wind energy and as in this case on the avian risks of coal as an energy source. But this is not the only place where this paper makes an appearance. The Wikipedia article on the environmental impact of wind power also presents the numbers from this report as the authoritative numbers for use especially in the new metric “Estimated deaths per Gigawatt/hour (GWh)”. 

The Sovacool paper seems to be a go-to source for any number of activists and the deeper I read into the paper the more confused I get about how it has gained such prominence. For me, however, it represents a clear demonstration of the weakness of the peer-review process. To be clear, this paper has undergone peer review. Moreover, the peer-reviewers provided comments which the author incorporated into the final product (as described here). My problem is that even after peer reviewer the numbers coming out of this paper appear suspect. To explain my feeling let’s spend a bit more time examining the numbers that are derived from it. 

I initially discussed this paper in my blog post Wind Energy and Avian Mortality: Why Can't We get any Straight Numbers? As I discuss in that post, Dr. Sovacool does a lot of interesting things in this paper. To be clear, nothing he does is unethical. Moreover, he explains his assumptions and has a rational explanation for each of his decisions and extrapolations. My problem is that in virtually every case, his decision/choices would differ from how I would approach the problem. As an example, in my earlier post I break down how he estimates avian mortality associated with wind farms. In the post I demonstrate that using other data that was readily available at the time, entirely different (and much higher) estimates of bird deaths attributable to wind farms can be derived. As I describe in my earlier post, my approach mirrors that of many other organizations and the numbers I generated were consistent with those produced by other organizations and were in line with Canadian numbers generated for a special edition of Avian Conservation & Ecology (ref). This is not to say that Dr. Sovacool’s numbers are wrong, just that they are much lower than those derived by virtually every other researcher who has worked in the field. Having dealt previously with his approach to the problem of avian mortality and wind energy, the remainder of this post will look at how he approached avian mortality and nuclear energy and coal facilities. 

With regards to nuclear energy Dr. Sovacool derives an avian mortality rate of 0.416 fatalities per GWh, which he compares to the lower mortality rate he calculates for wind energy (discussed previously).  Looking at his analysis, over half his mortality rate nuclear power is derived from uranium mining and milling processes (0.228 GWh). As I discussed previously, this is the first area where I was confused. In his appraisal of wind energy he included only the installed wind turbines without consideration for the mining of rare earths for the wind facilities or for avian deaths attributable to the transmission lines. To derive the mining and milling number for nuclear power he uses two sources. His first is the Canon City Uranium Mill in Calorado which was fined for the death of 40 geese and ducks in 2008. He then uses that one-year number as an estimate for the annual number for the facility (quantified to 0.006 deaths per GWh). His second source involve some open pit mines in Wyoming (not uranium mines mind you but from uranium-bearing formations) which formed pit lakes where selenium (not uranium) associated with the ore leached into the water making them toxic to birds. He then estimated that the high selenium concentrations could kill approximately 300 birds a year (a rough guess and a proxy as he describes it). Using the pit lake results and his “proxy” he then corrects for a much bigger mine size to give him a final number (0.45 deaths per GWh). To generate his mining and processing number he averages the one-off value of 0.006 and the “proxy” + extrapolation number of 0.45 to calculate an average of 0.228 avian fatalities per GWh. The average of these two sources (one of which isn’t even a uranium facility) are then extrapolated for the entire nuclear industry in the United States. 

The other component of Dr. Sovacool’s nuclear estimate is based on collisions with nuclear plants (especially cooling towers). To derive this number Dr. Sovacool averages rates generated from four sources. As described at this web site (ref) and confirmed by Dr. Sovacool here (ref) the largest of the four numbers (representing approximately 60% of the combined total) is derived from strikes to cooling towers of an associated fossil fuel plant and not a nuclear facility. Once the data is corrected to reflect the nuclear power plants only (and still relying on Dr. Sovacool’s other numbers) the number decreases from an average of 0.269/GHh to around 0.07/GWh (ref). So restricting ourselves to only the actual nuclear facilities and in all other ways following Dr. Sovacool’s procedures, it is possible to derive an avian mortality rate associated with nuclear of 0.076/GWh. This represents less than 20% of Dr. Sovacool’s reported number and still is likely high based on the choices made in the paper.

The most fascinating case in the paper has to be its derivation of a coal avian fatality rate. Once again he starts by developing an estimate for the upstream effects of coal mining. He starts by looking at mountaintop removal and valley fill operations in four states and considers how the loss of habitat from these facilities will affect Cerulean Warblers (0.02 warbler deaths per GWh in those states). He then extrapolates that number for US coal production. This ignores the fact that approximately 60% of coal mined in the US is from underground mines (ref) and that mountaintop removal and valley fill mines don’t even represent the majority of the coal mined in some of the states where it is still legal (ref). He then looks at collisions with power equipment and averages two studies, one of transmission line strikes in Spain and a second of strikes on the Kincaid Power Plant in Illinois. Using these two locations he derives a mean number for all plants across the United States. Notice how in this case, and this case alone, he includes transmission lines in his calculations. Presumably transmission lines are not necessary for wind installations or nuclear facilities. Moreover, the study he chose to use was not even in the United States, yet it and results from a single other plant were used to estimate U.S mortality numbers (0.07 deaths/GWh). Surely there exits data from more than a single plant in the US, but if that data exists, it was not used in this paper. 

Having looked at the mining and processing (0.02/GWh) and the operation of the units (0.07/GWh) Dr. Sovacool then does something he does not do for other power sources and looks at off-gases from combustion. In this case he derives numbers for acid rain and mercury effects. I lack the patience to go into details about how he derives these numbers, but I welcome anyone with the patience to note that he extrapolates national numbers from acid rain estimates in the Adirondacks and for mercury effects he includes reports that involve decreasing penguin populations (yes he does specifically acknowledge penguin data is used in the derivation of his US numbers). Using these tools he manages to obtain an estimate of approximately 0.11 avian deaths/GWh. So using most of the arrows in his quiver Dr. Sovacool derived a rate of 0.20 deaths per GWh. This number would appear to be clearly exaggerated but was insufficient for this paper and so the last arrow was drawn, that of climate change. In this case Dr. Sovacool relied primarily on a paper in Nature (ref) that presents hypothetical scenarios for bird extinction depending on a number of future climate change scenarios. Using the maximum hypothesized extinction rate for the worst-case scenario presented in that paper he derived (without further explanation) a rate of 4.98 deaths/GWh? Adding the 0.2 deaths/GWh from the other sources with the 4.98 deaths/GWh attributable to “climate change” he derived a rate of 5.18 fatalities per GWh which he then extrapolated to approximately 14.1 million birds a year. It is unclear how the USN report managed to reduce that number to 7.9 million birds a year but once again I lack the energy at this point.

You can probably tell that the effort to examine this paper has tired me out (reading all this has probably tired you out) but going through this painful effort demonstrates just what kind of numbers the activists rely on to make their points. It is a real pity that given the tremendous amount of high quality information existing in the peer-reviewed press, that the activists continue to go back to the same dog-eared papers. They almost always argue that these papers are peer-reviewed and must therefore be reliable. However, as my examination of this paper demonstrates, even when a paper is peer-reviewed, the results might be questionable. Peer review typically only ensures that the work is reproducible and in this case Dr. Sovacool is scrupulous to define all terms and explain his choices. In this, his work is utterly reproducible. The problem with this paper is that in each case you can look at the choices he made and have to ask yourself why he made that choice?