At its annual meeting in Amsterdam, on March 23, the InterAcademy Council (IAC), a multinational body of various national science academies, unanimously agreed to conduct an independent review of the processes and procedures followed by the Intergovernmental Panel on Climate Change (IPCC) in the preparation of its assessment reports (ARs) on the state of knowledge on climate change, its causes and its impacts. Established by the United Nations Environment Programme (UNEP) and the World Meteorological Organisation (WMO) in 1988 to provide the world with a clear scientific view on the current state of climate change and its potential environmental and socio-economic consequences, the IPCC has to date brought out four ARs: in 1990, 1995, 2001 and 2007. These reports, which have been endorsed by the worlds governments, have provided much of the scientific basis for the global climate policy framework, in particular the U.N. Framework Convention for Climate Change (UNFCCC).
Open to all members of the U.N. and the WMO, the IPCC is a scientific body comprising thousands of scientists from all over the world that reviews and assesses the most recent scientific, technical and socio-economic information produced worldwide on climate change. It does not conduct any research nor does it monitor climate-related data or parameters. The scientists producing the ARs also work on a purely voluntary basis.
Because of its scientific and intergovernmental nature, the IPCC is designed to provide rigorous and balanced scientific information based on robust analyses of available climate-related data and research findings as an aid to the decision-making process of different governments. The work of the IPCC is, as its website says, policy-relevant and yet policy-neutral, never policy-prescriptive. The IPCC has already initiated the process of preparing the Fifth Assessment Report (AR5), which is expected to be completed in 2013-14. The nominations for the 3,000-odd experts who will be involved in the exercise have already been received. Of these, about 600 to 700 will be the lead authors and the others contributing authors and review experts. The selection process is likely to be completed by the third week of May, and the AR5 exercise will begin after the IPCC plenary in October 2010.
The IAC was founded in 2000 to mobilise scientists and engineers from around the world to provide evidence-based advice to international bodies. It, therefore, seems somewhat odd that the IAC should actually undertake the scrutiny of another international scientific body, which is much more than its stated objective of just giving evidence-based advice. The IAC review is actually consequent to a joint request made by U.N. Secretary-General Ban Ki-moon and R.K. Pachauri, the Chairman of the IPCC, to the IAC on March 10. This was a response to the mounting criticism of the IPCCs functioning following revelations of the glaring errors in AR4, in particular the erroneous statement on Himalayan glaciers in the report of Working Group II (Impacts, Adaptation and Vulnerability). This statement was based on information from non-peer-reviewed, or grey, literature. It occurred in Chapter 10.6.2 of the report. A part of the paragraph was also repeated in Box TS.6 of the WG II Technical Summary of AR4.
The statement said: Glaciers in the Himalaya are receding faster than in any other part of the worldand, if the present rate continues, the likelihood of them disappearing by the year 2035 and perhaps sooner is very high if the earth keeps warming at the current rate. Its [sic] total area will likely shrink from the present 500,000 to 100,000 km2 by the year 2035 (WWF, 2005) (emphasis added).
The errors, and the non-peer-reviewed sources were brought to light by James Cogley of Trent University, Canada, through his comments in a BBC news story in December 2009 where he described it as wildly inaccurate and in a letter to Science in January. Without going into the entire history and anatomy of how this wrong information came to be included in AR4, one can see that this paragraph contains a combination of errors and inconsistencies and is written in bad English a result of its being a cut-and-paste job and the failure to trace the information back to the primary sources and verify it. In particular, the year 2035 seems to have originated from a 1996 document of the United Nations Educational, Scientific and Cultural Organisation (UNESCO) by V.M. Kotlyakov where 2350 was roughly given as the year by which world glaciers and ice caps, totalling an area of 500,000 km2, could shrink to 100,000 km2.
It seems that a media report wrongly used this area value while describing the Himalayan glaciers and also wrongly copied the year as 2035. This found its way into a report of the World Wide Fund for Nature and was taken verbatim from it for the AR4 WG II report without the original sources being cross-checked. The statement also escaped the verification process at various levels, including the lead authors who finally approve the document, as per the laid-down procedures of the IPCC. Further, the use of the words likelihood and likely in italics acquired a quantitative meaning because these words have been defined in statistical terms in the AR4. Fortunately, this erroneous statement did not form part of the Summary for Policymakers or the Synthesis Report, the concluding document of the AR4.
However, the Himalayan glaciers gaffe got reported and commented upon extensively in the media and in publications, and the IPCC came under severe criticism. There were other errors as well, though somewhat less serious, which were also caused by information being sourced from grey literature, but these somehow did not get the same kind of adverse publicity as the glacier-related report. One error was related to statements regarding the decline in rain-fed African agricultural output by 50 per cent during 2000-2020. Similarly, a statement on the proportion of the Netherlands below sea level, and therefore under threat of permanent inundation as a result of sea level increase due to climate change, gave the figure as 55 per cent. In this case, however, the source seems to have been data supplied by the Netherlands environmental agency, a body of the Dutch government itself. But this too did not get cross-checked for accuracy and authenticity.
These inaccuracies were, on the one hand, providing fresh fodder for climate sceptics and, on the other, threatening to undermine the credibility and scientific integrity of the IPCC, which governments and the public see as the most authoritative scientific body to assess climate risk. Besides charges of the IPCC being politically motivated to present a biased view of climate change science, the organisation came under severe attack for these errors and for its slipshod review and verification process before the ARs were made public and became the basis for policy actions of governments.
Faced with the barrage of criticism, the Chair and the Vice-Chairs of the IPCC and the Co-Chairs of the IPCC Working Groups issued a joint statement on January 20 withdrawing the controversial paragraph on Himalayan glaciers. The conclusions of the Synthesis Report that the current stresses on water resources, the widespread mass losses from glaciers, and the reductions in snow cover over recent decades would accelerate throughout the 21st century were robust, appropriate and entirely consistent with the underlying science and the broader IPCC assessment, the statement said. It further stated that the underlying assessment on Himalayan glaciers was based on poorly substantiated estimates of rate of recession and the date of disappearance of Himalayan glaciers. In drafting the paragraph in question, the clear and well-established standards of evidence, required by the IPCC procedures, were not applied properly.
In the wake of this episode, the IPCC has been criticised for using information and data from grey literature. The use of non-peer-reviewed literature to fill data and information gaps is actually permitted under the procedures laid down by the IPCC as long as there is stringent verification and validation of the information used. Annex 2 of the IPCC document on detailed procedures for producing reports describes the procedure for using non-published/non-peer-reviewed sources. It says: [I]t is increasingly apparent that materials relevant to IPCC Reportsare found in sources that have not been published or peer-reviewed (e.g., industry journals, internal organisational publications, non-peer-reviewed reports or working papers of research institutions, proceedings of workshops, etc.). Authors who wish to include information from non-published/non-peer-reviewed sources are requested to critically assess any source they wish to include . Each chapter team should review the quality and validity of each source before incorporating results from the source into an IPCC Report (emphasis added). It is interesting to note that the authors are only requested to adhere to the guideline on grey literature.
On February 2, the IPCC issued another statement reiterating the salient features of its principles and procedures. It said: The first component of an IPCC assessment is that all of the relevant literature is considered, whether or not it agrees with the dominant paradigms, and whether or not it has yet stood the test of time compared to other studies.The second component is that the information from all of this literature is distilled into key messages that capture the state of knowledge at the time of assessment. Theseneed not be known with 100 per cent certainty, but the level of confidence must be carefully stated.... Most of the scientific literature assessed by the IPCC is published in scientific journals, where the journals peer-review and editorial process provide an important foundation of quality control. Some important information appears not in scientific journals but rather in reports from governmental and non-governmental organisations. For the IPCC to fulfil its comprehensive assessment mandate, it needs to assess the information in these reports. This is an important responsibility, but it is also a challenge, because the diverse approaches to reviewing and editing these alternative sources of information force the IPCC authors, reviewers and review editors to utilise additional care and professional judgment in evaluating them.
Though the IPCC process was clearly at fault for not ensuring that its guidelines were steadfastly adhered to, the IPCC statement on the procedures adopted also correctly reflects the corrective mechanism that is intrinsic to the process of doing science.
Independent of whether a result is published in a high-profile scientific journal or a government report, the statement added, it is not the case that the conclusions in every publication turn out to be 100 per cent complete or correct. New data and new perspectives often reveal new mechanisms, complications or possible interpretations. Sometimes the investigators conducting the study make mistakes or misinterpret data.The essence of science is testing interpretations against observations, and gradually building a body of knowledge that is consistent with all of the observations and experiments. A careful assessment is a powerful tool for transforming a huge body of science into the kind of knowledge that can support well-informed policy choices.
Asked whether the IPCC would continue to rely on grey literature, Pachauri said: There is not much research being carried out on climate issues in several parts of Africa. We have to rely on non-peer-reviewed literature in such cases. However, in the case in question, the laid-down procedure was not diligently followed. We have now emphasised the need for the lead authors and the Co-Chair to carry out reviews of such literature proactively. Defending the controversial statement in AR4 on African agriculture, he said that there were robust reasons to make that statement. It mentions up to 50 per cent decline; one is not talking of average here, he added. On the Himalayan glaciers issue, he said that there was no adequate research, particularly mass balance studies, to address the question that is commensurate with the extent and massiveness of Himalayan glaciers.
The Chinese are doing extensive research on their side of the Himalayas, he noted. But the errors that have crept in highlight the greater perils of using grey literature from activists groups as against that from organisations engaged in genuine research, governmental or non-governmental. This distinction, it would appear, is particularly important for the IPCC, given its fundamental role in shaping nations climate policies.
In a bid to counter the growing current of public scepticism against climate change in the wake of controversies around the IPCCs functioning, over 250 scientists of the United States issued an open letter on the IPCC and the errors contained in AR4. Many in the popular press and other media, as well as some in the halls of Congress, are seizing on a few errors that have been found in AR4 of the IPCC in an attempt to discredit the entire report. None of the handful of misstatements (out of hundreds and hundreds of unchallenged statements) remotely undermines the conclusion that warming of the climate is unequivocal and that most of the observed increase in global average temperatures since the mid-twentieth century is very likely due to observed increase in anthropogenic greenhouse gas concentrations. Despite its excellent performance for accurately reporting the state-of-the-art science, we certainly acknowledge that the IPCC should become more forthcoming in openly acknowledging errors in a timely fashion and continuing to improve its assessment procedures to further lower the already very low rate of error. It is our intention in offering this open letter to bring the focus back to credible science, rather than invented hyperbole, so that it can bear on the policy debate in the U.S. and throughout the world.
With regard to the last statement above, the IPCCs retraction of its statement on Himalayan glaciers would still seem to fall short of admitting the error openly and totally. Moreover, it fails to even acknowledge the errors (of judgment or otherwise) of the statements pertaining to African agriculture and land below sea level in the Netherlands. Notwithstanding Pachauris defence on the matter, as the scientists letter points out, caveats that were carefully crafted within the chapters were not included when language was shortened for the Synthesis Report.
In any case, the letter stated, it is essential to emphasise that none of these interventions alter the key finding from the AR4 that human beings are very likely changing the climate, with far-reaching impacts in the long run. Recent events have made it clear that the quality control procedures of the IPCC are not watertight, but claims of widespread and deliberate manipulation of scientific data and fundamental conclusions in the AR4 are not supported by the facts. We also strongly contest the impression that the main conclusions of the report are based on dubious sources. The letter also urged the IPCC to put an erratum on its website that rectifies all errors that have been discovered in the text after publication.
The proposal to engage the IAC for a review, which was endorsed by the Executive Director of the UNEP and the Secretary-General of the WMO, should be viewed from the perspective of IPCCs principle of being open, objective, apolitical and transparent. In their letter to the IAC Co-Chairs Robert Dijkgraaf, president of Royal Netherlands Academy of Arts and Sciences, and Lu Yongxiang, president of the Chinese Academy of Sciences Ban Ki-moon and Pachauri said: In recent months, a very small number of errors have been brought to light in AR4 of the IPCC, a document containing thousands of peer-reviewed and independent scientific studies. However, the bedrock scientific consensus on climate change as described in AR4 remains unchanged. Given the gravity of the global threat posed by climate change, it is vitally important to ensure full confidence in the scientific process underpinning the assessments of the IPCC. As the IPCC embarks on its AR5, it is imperative that its work be as accurate, objective, comprehensive and transparent as possible, and that the potential for any future errors is minimised.
The IAC was also asked to establish an ad hoc Independent Evaluation Group (IEG) of experts from relevant fields to conduct the review and to present recommendations on possible revisions of IPCC procedures and practices. According to the IAC release of March 10, the IEG will also be asked to recommend measures and actions to strengthen the IPCCs capacity to respond to future challenges and ensure the ongoing quality of its reports. In his remarks to the U.N., Dijkgraaf said that the proposed terms of reference (TOR) are
Data quality assurance and data quality control at the IPCC;
Guidelines for the types of literature appropriate for inclusion in IPCC assessments, with special attention to the use of non-peer-reviewed literature;
Procedures for expert and governmental review of IPCC material;
Handling of the full range of scientific views; and
Procedures for correcting errors identified after approval, adoption and acceptance of a report.
We have also been asked to analyse the overall IPCC process, including the management, administration and transparency of the IPCC, Dijkgraaf said. In this study, as in all our work, the IAC will conduct its review independently according to the IACs own procedures. Neither the U.N., the IPCC, the WMO, nor the UNEP will exercise any oversight or control over the study process or our final report, he added.
Following the decision to carry out the review, the IAC will soon constitute the IEG. India, being one of the 15 countries whose academies are represented on the IAC board, will play a key role in the process of selecting the IEG, said M. Vijayan, president of the Indian National Science Academy (INSA). Whether any Indian scientist will be part of the IEG, one cannot say, he added. According to Dijkgraaf, the IAC has been assured of financial support from the U.N. for this study, but the scientists of the IEG will not be paid and only their travel and meeting expenses will be met by the IAC.
Interestingly, on March 22, Jairam Ramesh, Minister of State for Environment and Forests, sent a seven-page letter to Dijkgraaf highlighting areas related to the IPCCs working in which improvements can and should be made. In particular, the Minister said: It has also been observed that inbreeding in the form of a particular type of information is repeatedly published by a single group or group of authors adding to the number of references as the sole source. There is complete absence of alternative literature/information by others. It stands to reason that the work of an acknowledged authority a single author or a group in a particular area is likely to be published more often than that of others. Accordingly, the work of an acknowledged authority will also get repeated citations and be repeatedly referred to. Similarly, the procedures in place at the IPCC ensure that if alternative views do get published in peer-reviewed journals, they will be given equal consideration.
So, the point of this statement by the Minister is not very clear. It is clear from the TOR of the IAC that all the points that the Minister raised in his letter have been duly taken care of. Though the letter was received only after the IAC meeting was over, on March 23, Dijkgraaf has apparently taken note of the letter, and it will be discussed by the IEG once it is formed. Interestingly, India was the only country to have sent such a letter to the IAC.
Because the organisational work for AR5 has already begun, the IAC has been required by the TOR to submit its report to Ban Ki-moon by August 31 so that there is a document ready for consideration at the IPCC Session in October 2010. According to Pachauri, it is not necessary for all the recommendations of the IAC to be adopted by the IPCC. Once the IAC report is submitted, it gives us eight weeks for all the governments to consider the report in advance of the October Session and to be prepared to decide on actions that need to be incorporated in our guidelines and become part of the review of AR5, he said.