Sunday, February 15, 2015

Is the IPCC the IOC of Science?


I was having a discussion online, on Twitter, about the field of Climate Science. The basis of the discussion was a simple question: what does it take to be considered a "climate scientist"? and the corollary: who is qualified to comment on and provide reviews of climate science? As many of you may know, I have taken flack for suggesting that highly educated and well-trained individuals outside the academic sphere have anything to say about the topic and deserve the opportunity to be heard. The debate went back and forth with people pointing out that climate science is a complicated field and methodologies used in climate science are too complicated for input from the general public. What no one was able to actually define was: what it took to be called a "climate scientist". My opinion, coming out of the discussion, was that climate science is not really "a field" as much as a group of fields centered on answering questions about several inter-related topics. During the course of the discussion a thought occurred to me. If I had to describe the field of climate science in a way that best expresses its current status in the world, I think the best analogy would be the various sports that make up the Olympic Games. With that thought in the back of my mind, I went out for a run and as the kms flowed underfoot, I became more certain that this was a readily-accessible way to make people understand what climate science is all about. 

In my view, climate science is not a "subject", per se, but rather a field made up of a number of disparate subjects, many of which had few interactions prior to the creation of the field of “climate science”. A population ecologist studying the effects of climate change on woodland bird populations has pretty much nothing in common with an aerosol physicist, but under the umbrella of "climate scientist" they can be considered to be working in the same field. They do share some very important similarities in that both are highly educated, having reached the apex of their respective fields, but their methodologies and field techniques have about as much in common as an athlete in the modern pentathlon has with a beach volleyball player (no opinion on which is which). On a day-to-day or week-to-week basis these people do not interact, but once every few years they are asked to do their best for an international audience. 

The more I thought about it, the more intriguing this analogy became. The Olympic Games bring together the world's greatest specialists, in their individual sports, while the IPCC brings together the top scientists in their fields. Both then ask those specialists to perform under the eyes of the world. Consider that the IOC is a body started with the best of intentions that was supposed to be above politics. Over time, however, it has been a lightning rod for politics. Does this sound like the IPCC? The membership of the IOC includes some of the top athletes and athletic officials on the planet. It is also the home for numerous has-beens, never-weres and political hacks. Remind you of anything? It has been argued that in the last 50 years the IOC has become a fiefdom controlled by a small body of bureaucrats who are accountable to no one but themselves? See the similarities? It is not all bad, however, as the Olympic Games are one of the premiere sporting events on the planet. They represent a showcase for some of the planet’s greatest athletes and are home to some of the best officiated sports on the planet. But the Olympics, due to their rules allowing inclusion of all member states, were also the place where "Eddie the Eagle" demonstrated that he was not quite ready for prime time, while officials in figure skating were accused of colluding to ensure that their respective athletes got on the podium, regardless of how they performed on the ice. Most importantly, while it is possible to respect the athletes who have spent years perfecting their craft, it is also incredibly easy to get turned off by the ugly politics and the crass behavior of the bigwigs who have made a home in this unaccountable organization. 

Going back to the citizen-scientist question, as a young man I used to be a runner. In University I ran track and field, with my event being the steeplechase. I had the benefit of truly excellent coaching and was even given the opportunity to coach track for several years, a couple of which I did under another great coach who taught me the intricacies of the various sports. Now I was never going to be an Olympic athlete, as I lacked the genetic gifts to be the very best, but I know more about track and field than most people in my community. I have also served as a running instructor/coach and have taught many hundreds of people how to run. When I watch high-level athletes, I can appreciate their skill and given time and slow-motion film, I can break down a steeplechaser’s stride and figure out whether they are dragging their following foot over the hurdle. Were I to do so, I could even give advice to an athlete who is much more talented than me. Their reply could range from: “who are you? I already have a coach” to “thanks for the info, I will look at the films and see if you have caught something that my coach missed”. Similarly, I can watch a high-hurdler or a 400 meter hurdler and identify issues with their technique (if an issue exists). In a similar vein, Steve McIntyre may not be a “climate scientist” but his knowledge of data analysis far exceeds that of many academics and when he suggests a statistical analysis has a fault, I would suggest listening very carefully.    

So where am I going with all this? Well while not all of us can be Olympic athletes a lot of people out there have the knowledge and skill to assess their efforts. Meanwhile, while you might be a “climate scientist” you may have no clue of the strengths and limitations of another “climate scientist’s” work. Moreover, it is likely that even though you are a “climate scientist” that someone else, often outside the academic sphere, knows more about topics outside of your specific area of expertise, than you do. Finally to return to our analogy, both Usain Bolt and Ben Johnson are technically Olympians, but we hold one in much higher esteem than the other. Similarly, it is possible to hold some climate scientists up as examples for the next generation of scientist while recognizing that not all climate scientists meet those lofty standards.

9 comments:

  1. Blair, this is an excellent analogy. If I may add, I noticed those who try to place "climate science" in an inexpugnable fortress are not "hard scientists". Did you check your interlocutors' backgrounds? I find beacoup columnists and bloggers who don't understand multidisciplinary discipline approach. They just haven't been exposed to it.

    I've been "Assurance Manager" for a large multinational's subsidiary, this required I make sure what was being done "passed the smell test". And I could get diagnostic information out of dynamic models simply by requesting history match sampling. The GCMs don't pass the smell test for regional scale critical predictions. And as I used to say, it's my judgement. It has little to do with climatology.

    ReplyDelete
  2. Hi Fernando (and Blair). I feel referred to, and just wanted to point out that while I am not a scientist, my specialization is in helping citizens give policy input in questions where science is involved. My position is that when they are trying to determine where the science is pointing in any regard, on a probabilistic basis they are likely to get the soundest answers from research scientists in the field in question. It is true that as Blair set there will be meritocratic differences among them and individually or collectively at times they might be off in a direction that will later be shown to be wrong. But there is nonetheless, on balance, no other set of persons that can be identified whose aggregated positions are more likely to guide them as well as that set is. There are numerous reasons for that including their training and experience, the way the scientific communities are designed to investigate and replicate each others' findings, as well as the fact that their careers and reputations depend on their credibility whereas the rest of ours don't. Again, there are times when this type of reliance on expertise may fail citizens, but it seems highly probable to me that on balance it will save them far more often.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. I should say "try to replicate."

      Delete
    3. Agreed, but there are also situations where the investigators are too close to the topic or lack information that is available to outsiders, or specialists in other fields. This was clearly demonstrated in East Anglia, where the statistical expertise of the group was sadly lacking. This resulted in errors that could not be identified therein but were clear and readily apparent to individuals working in other fields who depended on those statistical techniques for their livings. In an Interdisciplinary field like climate change, the presence of non-academic specialists should be considered a bonus as it supplies much needed expertise and specialization not always available in the academic sphere. Instead the academics treat these highly trained and motivated individuals as a challenge or a hindrance to their work because the outsiders expect the same measure of expertise from the scientists as they would from their colleagues in the private sector.

      Delete
    4. Suzy, my experience is exactly the opposite. Don't forget my job involved judging the quality of the work performed by others. I think it's possible you just haven't seen people like me, who had to undergo approximately 20 years of training and education after graduation before I was promoted to what we jokingly called "risk policemen". I'm not a scientist, I'm a highly trained engineer trained to take apart studies, research proposals, recommendations, budgets, and also judge whether individuals were qualified to do some jobs. My personal experience tells me scientists need shepherding and policing.

      By the way, I have a sister who did research at "A Prestigious Medical Research Organization" and based on what she told me they definitely need trained outsiders to police their work. It's just human nature, Homo Sapiens just gets sloppy, especially if it thinks it belongs to an all powerful group. And no, peer review doesn't help either. Sometimes one has to get incredibly hard nosed to extract what really goes on.

      Delete
  3. The IPCC is nothing like the IOC.

    For instance, I know nothing about running, but I can see that Bolt is the fastest. I know about as much about chemistry, but I cannot see who is right in a debate between chemists -- I would have to rely on imperfect indicators such as track record, pedigree and body language.

    There is another difference. The National Olympic Committees by and large submit their best athletes. The National Representations of the IPCC often select people on criteria other than expertise.

    ReplyDelete
    Replies
    1. Richard, your comment appears to reflect some misconceptions about the realities of International sports. While the Bolt example was imperfect consider the picking of members in team sports. Every Olympics there are complaints that the "best" athletes are left off the team in order to ensure the best "team" makes it to the games. Moreover due to rules of admission the best athletes are often excluded in order to allow more countries to compete. So even though 7 of the top 10 marathon runners in the world are Kenyan only three are allowed to compete in the Olympics. Meanwhile in 2012 in London Tsepo Ramonene from Lesotho was allowed to compete having only finished a single Marathon in his career. He finished nearly an hour behind the winner.

      Delete