II. 2 Working Group 1 Rapporteur’s Report

THE FORM OF ASSESSMENTS

Chair: Ari Patrinos

Rapporteur: James McCarthy

Members: Daniel Albritton, Shardul Agrawala, Barbara Connolly, Elizabeth DeSombre, John Houghton, Robert Keohane, Ronald Mitchell, Edward Parson, James Risbey

This Working Group focused much of its discussion around four questions relating to the form of global environmental assessments:

How does context affect the goals of the participants in the assessment process?

What strategies seem most and least useful for negotiating assessment designs that balance scientific and political concerns in the design of assessment processes?

1. CONTEXT AND GOALS OF ASSESSMENTS

Purposes of assessments and goals of individuals involved in environmental assessments range greatly. In some instances the purposes may relate to an advocacy position of a sponsoring body, and participants may be selected, perhaps self-selected, for their sympathy with the advocacy position. For environmental assessments undertaken by governmental and intergovernmental bodies the common purpose is to provide a comprehensive and objective review of scientific understanding with appropriate statistical qualifications regarding estimates of uncertainty. Objectivity of assessments can be ensured by broad representation of sponsoring and expert groups in designing the assessment and vetting its products.

While most, if not all, scientists involved in environmental assessments would expect their efforts to be useful in some aspect of the decision making arena, some assessments are generated without clear prior knowledge of a receptive decision making community. Conversely, the utility of an assessment in decision making processes is likely to be a primary objective of a sponsoring advocacy group that has as its purpose the promotion of specific views regarding environmental change. For the most part, the types of assessments considered in the following discussion are grounded in the natural sciences. Our discussion group's attention was principally directed towards assessments that have demonstrable objectivity with respect to selection of participants and include peer review of the assessment process and/or its products.

In the course of our consideration of different environmental assessments, we found evidence to support the view that the more inclusive the ensemble of sponsoring/authorizing bodies, the more likely it is that the scientific products of such an assessment will be widely known and respected in the decision/policy making community. There are clear examples of this on both the national and the international level. We also noted here and elsewhere in these considerations a feedback that we referred to as a "ratcheting" phenomenon. For example, given the widespread visibility of the IPCC assessments, it is less likely that any specifically national assessment of the underlying science of global climate change could have the same impact in the future that national assessments of global climate change once had, either nationally or internationally.

Broad inclusivity of the policy community will be appealing to some scientists and not to others. Some participants believe that their efforts to promote an objective understanding of an environmental issue have important added value if the sponsoring bodies are broadly representative of the decision making community. Broad political acceptance may, however, require inclusion of certain participants largely for political balance. Because of this, certain scientists may be less motivated to participate (the deliberations of a larger more inclusive group can be more cumbersome and less efficient). Other scientists may be stimulated to participate because they value such inclusivity, in that it fosters capacity building in critical regions/sectors. In this regard, capacity building can be an ancillary product of an assessment.

A contextually broad and successful end-to-end assessment demands that the goals of the scientific community not be limited to producing an assessment simply on the natural science aspects of the problem. The scope of such an assessment must also include a commitment to the evaluation of impacts and a contribution to the development of science based "if-then" scenarios (i.e., if this is done by the decision making community, then the following response in environmental conditions can be anticipated). However, this policy-directed portion of the assessment is most successful if it studiously avoids recommending specific policy actions, and instead evaluates the potential consequences of realistic policy options. While the context of an assessment may require participation of scientists in such scenario exercises, this may well be the most challenging portion of the entire assessment process for some scientists. The development of response strategy scenarios is only manageable with a small group of participants, and it can only be successful if a few scientific leaders of the assessment team have their peers' confidence to represent effectively the collective understanding of the entire team in this endeavor.

2. THE INFLUENCE OF CONTEXT ON SUBSTANTIVE AND PROCEDURAL CHARACTERISTICS

The context of an environmental assessment will reflect the broad interests of the range of participants in the assessment. The broader the context, the greater the potential for high level of policy community interest. An example of this would be the relative narrowness of political interest in a nationally based global environmental assessment vs. the broader political context of an assessment like that of the IPCC. Moreover, it is unlikely that a global environmental assessment with a very broad international base, such as the IPCC, would be supported by governmental and intergovernmental bodies unless it was designed to include information relevant to their policy processes.

One procedural aspect of an assessment process of obvious interest to the sponsors/authorizers relates to their expectations of their role in vetting the overall summary statements arising from the assessment. Thus, while the scientific scope and character of the assessment and statements of scientific understanding may be free from intervention by the sponsors/authorizers, the final procedural

step of producing summary statements for policy makers can be prolonged and contentious. In the case of serial or sequential assessments, another example of the previously observed "ratcheting effect" is evident. The interest of sponsors/authorizers in vetting policy summary statements is more likely to increase than to decrease in subsequent assessments. The line-by-line review procedure of the policy makers summary in IPCC '95, which was often in the news, was not part of the procedure of the IPCC '90 assessment. The decision to insert this procedural step in the assessment was made by the IPCC Bureau following the first IPCC assessment. Such political involvement, is in part, driven by the realization that, in the course of sequential assessments, the IPCC will become increasingly comprehensive and, in the particular case of the IPCC, increasingly policy relevant. This tendency is perhaps inevitable with an assessment set in such a broad political context for which earlier efforts were judged to have been influential in the policy arena.

Conversely, an institutionally narrow assessment, which would be viewed as having fewer political consequences or implications will likely have fewer or simpler procedural processes involving the sponsor/authorizer bodies. The political context of the assessment can clearly evolve, as in the case of the IPCC, and its evolution can result in altered or incremental procedural characteristics.

3. SCIENTIFIC CREDIBILITY AND POLITICAL LEGITIMACY

Scientific credibility is essential if an assessment is to be influential in the policy arena. It implies participation of scientists with impeccable credentials and a proper unbiased peer review process that legitimizes both the process and its products in the eyes of the relevant scientific community. Such credibility must be established early in the process in order to engage the very best scientists. If, in the course of an assessment, the quality of scientific leadership were to wane, the decline of an assessment's credibility would likely be accelerated by the withdrawal of first rate scientists and a likely failure to engage similarly statured replacements.

While the scientific reputation of the scientists leading an assessment may well ensure visibility of their efforts, there is no alternative for the peer review process in establishing credibility of an assessment within the scientific community. All who are familiar with global environmental assessments will know of examples of advocacy groups attempting to establish scientific credibility of their assessment efforts largely on the basis of the reputation of the scientists involved. Sometimes such assessments do have political influence but, in the long run, for any assessment to have an enduring impact, it must be vetted by unbiased peers.

In order for an environmental assessment to have influence in the policy arena, scientific credibility alone is insufficient - it must also have political legitimacy. While there may be short cuts, such as personal connections among scientists and policy makers that allow for political receptivity of an assessment with minimal process, the most influential assessments are typically formulated with balanced attention to the issues of both scientific credibility and political legitimacy. Thus in pre-assessment negotiations, it is particularly important that all representatives of all parties engaged in an assessment agree to procedures for establishing the essential qualities of an effective assessment from both of these perspectives. Without sufficient attention to these important details, there is substantial risk that the assessment will be orphaned by one of these two essential communities.

Would a scientist knowingly invest effort in an assessment that was known to be without bona fide political legitimacy? Perhaps, if the problem were seen to be one that could be resolved with time, but without a realistic expectation of attaining such legitimacy, most scientists who are without a personal agenda regarding environmental issues would probably doubt the value of their efforts. In general, a scientist's motivation to excel in an assessment endeavor is probably proportional to the assessment's political legitimacy.

Though it may seem obvious that the quality of an environmental assessment will be enhanced by establishing both the broadest political base of sponsorship and the highest caliber of scientific participation, there are obstacles to both that need to be considered when negotiating the form of an assessment.

When an assessment focus is global, the inclusion of multiple agency or national sponsors optimizes the political legitimacy and acceptance of the endeavor. However, the multiplicity of sponsors and authorizers can either enhance or diminish the attractiveness of participation for certain scientists, and especially the latter if the sponsoring body is an operational agency with little record of commitment to fundamental research. Moreover, although there is little doubt that a broader audience has potential to amplify the value of the scientific input by exposing it to a broader policy community, a larger number of sponsors/authorizers needs to be satisfied with the product, and diversity among them may signal increased difficulty in finalizing the summary conclusions of an assessment.

An effective way to broaden assessment acceptance in the policy community is to attain sponsorship of intergovernmental bodies, as in the case of WMO and UNEP with the ozone and IPCC assessments. Under the auspices of this partnership, participating nations engaged prominent scientists from their academic research sectors. Clearly, all participating parties add value to their investment by negotiating the highest caliber of scientific participation and the broadest degree of political sponsorship.

4. THE BALANCE OF SCIENTIFIC AND POLITICAL CONCERNS

The Role Of Irreversibility Of The Environmental Problem In Shaping Assessment Form

The appearance of irreversibility of an environmental problem judged to be important to human or other biological systems can alter the way scientists and policy makers structure an assessment. The undeniable irreversibility of the extinction of a species is clearly the reason this environmental problem arouses such widespread public concern. While extinctions and the creation of new species are the essence of biological evolution on earth, actions of Homo sapiens may at this time be causing a rate of extinction unequaled since the postulated global consequences of an asteroid impact in the Yucatan region 67My BP. From this particular example, it is evident, however, that irreversibility in and of itself is insufficient to promote strong policy actions. Considerable harm to the functioning of biological systems and its large associated social costs must also be demonstrated convincingly. While many scientists have proposed substantial potential long term costs of species loss, few in the policy making arena have called for strong action to preserve maximal species diversity.

Though irreversibility of other environmental problems may not have the same finality as a species extinction, the projected risks of such problems to biological systems may arouse sufficient concern to result in policy solutions. With respect to human health risk, it can be assumed that an appropriate time frame for ascribing an approximately irreversible quality to such an environmental problem might be one human lifetime. With ozone depletion in the Antarctic polar vortex, the consensus scientific view by the late 1980s was that without action, the continued release of CFCs to the atmosphere would have irreversible consequences for biological systems, and specifically for humans in the form of an increased incidence of skin cancer. Natural photochemical processes in the atmosphere do destroy CFCs, but only very slowly relative to the rate at which we were releasing these compounds to the atmosphere. This long atmospheric residence time and projections of increasing rates of release indicated that decisions made today about CFC production will impact humans multiple generations into the future. Such realization resulted in widespread concern among the public and certain influential politicians, and action was taken to minimize further ozone loss.

Another example is the potential risk associated with the release of long lived radionuclides in air, water and soil - the persistence or irreversibility of these materials and the health risks they pose together generate widespread public concern.

The immediate consequences of increases in anthropogenic releases of greenhouse gases are perhaps no more benign for biological systems than ozone depletion. The possibility exists that the consequences of such releases might even be a more significant factor in species extinction than changes in land use. Yet, there remains less public concern and political resolve with regard to the greenhouse gas problem. It is possible that the "artificial" quality of CFCs and toxic radionuclides in the environment, in contrast to "natural" substances like carbon dioxide and methane, contribute to the degree of passion these issues evoke in the minds of environmentally literate individuals.

Climate change is a complex issue. The underlying science is more intricate and the scenarios more uncertain than in the case of ozone depletion. Importantly, unlike the CFC issue, there is no obvious candidate for a practical, less harmful alternative to fossil fuels. Thus, while from some perspectives the fossil fuel-climate change problem meets the criterion of irreversibility with potential to cause serious harm to biological systems, including human health, it lacks the widespread sense of urgency characterized by the CFC-ozone issue. Unless an environmental assessment reflects both relevant scientific understanding as well as a set of cost/benefit-characterized policy options, an "irreversible = urgent" position risks rejection by members of the policy community.

The Role Of The So-Called Smoking Gun In Shaping Assessment Form

An example of the smoking gun is the definitive cause-effect linkage established for CFCs and ozone depletion in the stratosphere. Among the competing hypotheses to explain the Antarctic polar vortex ozone depletion, the hypothesis related to CFC chemistry was most consistent with data generated from aircraft and land based field campaigns.

Due to the vastly more complex matter of the greenhouse gas-climate linkage, it is unlikely that a comparably unequivocal line of evidence will emerge in support of this linkage any time in the near future. The 1995 IPCC report took a step in this direction with its statement regarding "the first attribution of observed climate changes as being due to some human influence". It now appears that certain observed regional shifts in climate are consistent with models that take into consideration both the greenhouse gases and atmospheric aerosols.

The form of an assessment is profoundly influenced by objectives that include such attribution; this is particularly true when the trend of the variable for which causality is implied is a property such as regional or global temperature. Such conclusions can only be based upon very sophisticated statistical analyses. When an analysis is so esoteric that only a few scientists are capable of judging the quality of the process, there is risk that some aspect of it, perhaps an aspect that is critical, will be less well supported in a subsequent analysis.

Unless scientists and sponsoring or authorizing parties participating in an assessment fully recognize the nature of this risk when the assessment is designed, the effectiveness of their partnership can be jeopardized when concluding statements are formulated. As with all scientific findings the proper expression of the conclusions of an assessment will include probabilistic estimates of uncertainty. Some participants in an assessment may well expect statements of "fact". However, the scientific integrity of the assessment requires that even the most definitive and authoritative statement of understanding at the moment of the analysis be properly qualified. Subsequent assessments, reflecting more current understanding, may contradict earlier conclusions. Such is the way knowledge progresses in areas as complicated as climate research.

APPENDIX

Additional questions raised in the working group discussion for which insufficient time remained to adequately address:

• How does the manner in which scientific uncertainty is handled influence the form of the assessment (the thin and wavering edge vs. the thick and stable core of knowledge)? Scientists become preoccupied with the former while policy makers need most to know about the latter.
• How do decisions regarding strategies for dealing with extreme events influence the form of assessments? It is statistically simplest to deal with trends in mean conditions, while climate extremes may cause the most disruption to social, economic, and ecological systems.
• How do perceived tensions among audiences (sponsors/authorizers) influence the form of assessments? Will scientists be more reluctant to invest their time and effort if the sponsors/authorizers are likely to have strong differences of opinion regarding the importance of a particular environmental issue?
• How does the judicious and timely release of an assessment's results flow based on the form of the assessment, especially as the assessment proceeds from sequential emphasis on credibility, manageability, and accountability? With maturity, expectations of timely and high quality results increase, causing possible strains between various participants in the assessment process.
• How does the paradox of increasing emphasis on global assessments, and the growing policy community interest in regional assessments, especially when the scientific capability at the global level is marginal and at the regional level rudimentary, at best, influence the form of the assessment? Scientists learn early in their education that stature in their community can plummet precipitously with premature conclusions that are later found to be erroneous.
• How does a genuine public concern for environmental quality and its influence on governmental and business positions vis-à-vis global environmental ethics influence the form of assessments? Scientists are often naive regarding the sensitivity of governments and businesses to public opinion on environmental issues, and thus sometimes reluctant to participate in assessments sponsored by such bodies.
• How does a tension or perceived tension between assessment obligations and research needs of the climate community influence the form of an assessment? Some scientists are frustrated by the seemingly repetitive need to demonstrate how little is known about the climate system. When climate assessments are undertaken seriously by scientists, these activities do detract from their primary responsibility to improve our understanding of the underlying workings of the climate system.