Bank involvement in pollution control and urban environment projects forms a significant share of a growing environmental portfolio (61% of a lending portfolio that has almost doubled since 1992). As investments in this area grow, it becomes increasingly important to develop quantitative measures of the effect of such investments on the environment, in this case air and water. There is, therefore, a heightened need to use environmental performance indicators (EPIs) for monitoring success of investments in meeting the stated objective of pollution management.
Environmental Performance Indicators
An indicator is 'something that provides a clue to a matter of larger significance or makes perceptible a trend or phenomenon that is not immediately detectable' (Hammond et al, 1995). An indicator's main defining characteristics are that it quantifies and simplifies information in a manner that promotes the understanding of environmental problems, to both decision-makers and the public. Above all, an indicator must be practical and realistic, given the many constraints faced by those implementing and monitoring projects.
EPIs can help quantify impacts and monitor progress. The goal is to assess how project activities affect the direction of change in environmental performance, and to measure the magnitude of that change. Indicators that allow a quantitative evaluation of project impacts are particularly useful, since they provide more information than just whether the project is improving or degrading the environment. Information on the magnitude of a benefit is required to determine whether it is worth the resources being expended to achieve it. Similarly, information on the magnitude of adverse impacts might indicate whether the harm is justified given other benefits of the activity/project in question.
In the past, monitoring of Bank projects has focused on the inputs (resources provided under the project) and outputs (the immediate goods or services provided by the project). Input indicators can be specified in terms of overall funds earmarked, specific tasks to be funded, (such as new equipment, training), and funding agencies (IBRD, national government). Output indicators relate to specific actions taken (such as electrostatic precipitators installed, rehabilitation of water supply network, introduction of substances with low or zero ozone depleting potential, switching the fuel used in power plant) and these would evolve from the design phase of the project. As well as often being unduly rigid, such a project-centric approach also focuses attention too narrowly on the process of implementing projects rather than on its results. Increasingly, it is being realized that the ultimate assessment of the performance of a pollution related project should be based on the immediate and longer term effects on parameters such as air and water quality. The emphasis is therefore moving to the definition of outcome indicators (to measure the immediate results of the project), and impact indicators (to monitor the longer term results). While the input and output indicators relate more to project process, these outcome and impact indicators relate to the overall effect on the environmental resource, such as the quality of an airshed or water body.
For example, a loan to control dust emissions from cement plants could specify the following input indicators (to monitor the project-specific resources to be provided), output indicators (to measure goods and services produced), outcome indicators (to measure the immediate results of the project), and impact indicators (to monitor the longer term results).
input: financial ($X million) and/or technical assistance
output: number of electrostatic precipitators and fabric filter systems installed
outcome: reduced PM10 emissions measured in µg/Nm3
impact: reductions in ambient concentrations of PM10 in industrial center measured in µg/Nm3, or reduced health problems from respiratory diseases
Outcome and impact indicators should form an integral part of assessing the success of an environment sector project. Formulating effective outcome and impact indicators, however, is still a major challenge.
Considerable work has been done to come up with a coherent framework within which to assess the positive or negative effect of human activity on the environment. A conceptualization in terms of pressure, state, and response indicators, was developed by OECD (OECD, 1994). In this framework, three different aspects of the environmental problem are distinguished: the pressure that causes the problem (for example, emissions of SO2); the resulting state of the environment (for example, ambient concentrations of SO2 in the air); and the response to the problem (for example, regulations requiring the use of low-sulfur coal to reduce SO2 emissions and ambient levels). The pressure and state indicators measure project outcomes and impacts, respectively.
The pressure variable describes the underlying cause of the problem. The pressure may be an existing problem (for example, soil erosion in cultivated uplands, air pollution from buses) or it may be the result of a new project or investment (for example, air pollution from a new thermal power plant, loss of a mangrove forest from port development). Whatever the cause, pressures affect the state of the environment and then may elicit responses to address these issues.
The state variable usually describes some physical, measurable characteristic of the environment. Ambient pollution levels of air or water are common state variables used in analyzing pollution (for example, particulates concentrations in mg/Nm3 of air, BOD levels in the water body). For natural or renewable resources other measures are used: the extent of forest cover, the area under protected status, the size of an animal population, or grazing density. Most EPIs relate to easily measured state variables.
The response variables are those policies, investments, or other actions that are introduced to solve the problem. Bank projects that have important environmental components can be thought of as responses to environmental problems. As such, they can affect the state either directly by way of ex-post clean-up activities or by acting on the pressures at work (for example, by providing alternative income sources for farmers who would otherwise clear forests). In some cases, projects also seek to improve the responses to environmental problems (for example, by increasing the institutional capacity to monitor environmental problems and enforce environmental laws). Because Bank projects are themselves considered to be responses to environmental problems, the following discussion focuses on the use of pressure and state indicators to monitor project outcomes and impacts.
The relevant question is: What immediate and long term impacts are the project going to have on causal factors (pressures) and the condition (state) of the environmental problem? It is important to look at immediate outcomes that reduce pressures as well as the longer term impact, otherwise the project may be incorrectly blamed (or credited) for a worsening (or improvement) in the state of the environmental resource.
Choosing Environmental Performance Indicators
Choosing appropriate EPIs is a difficult task. No universal set of indicators exists which would be equally applicable in all cases. The diversity of environmental problems, of the contexts in which they arise, and of the possible solutions to them is simply too great. This section discusses how task managers might proceed to select EPIs for their projects and the factors that must be borne in mind when doing so. Given the limited experience in this field, the discussion is necessarily preliminary and likely to be revised based on the lessons of actually applying EPIs.
Link to project objectives. The process of selecting EPIs must necessarily start from a precise understanding of the environmental problems being addressed and of project objectives. Vague or over-broad objectives such as 'reducing erosion' or 'protecting biodiversity' are of little assistance in selecting EPIs (and may well indicate that the project or component itself is not very well thought-out). The appropriate responses will differ depending on whether erosion is caused by deforestation or inappropriate farming practices, for example, and so will the EPIs. Likewise, it makes a difference whether erosion is a concern because of sedimentation in downstream reservoirs or because it undermines agricultural productivity. Again, different EPIs will be best suited to each situation. Where the environmental consequence is not an explicit project objective but a by-product of project activities, the Environmental Assessment (EA) process can help to understand the possible impacts and hence to select appropriate EPIs.
Pressure vs state indicators. The goal of EPIs is to monitor and evaluate environmental impacts arising from Bank-supported activities. This implies a need to measure two dimensions of the environmental problem. First, the state of the environment and any changes in that state must be monitored. Second, the contribution the project is making to that change - both directly and indirectly - must be measured. Indicators of both pressure and state are typically required, therefore, to properly evaluate the impact of projects. Indicators of pressure alone are often insufficient because the link between a given pressure and the consequent effect on the state of the environment may be ambiguous or of unknown magnitude. An important factor in the design or assessment of a project is to determine as accurately as possible the relationship between the project and the overall state that is of concern. For example, airshed modeling may be required to quantify the relationship between a particular point source and ambient air quality.
Level of measurement. Indicators of state and pressure can both be measured at various levels. The objective of quantifying project benefits (or costs) will be aided if indicators are selected as close to the project objective as possible. This is particularly true when the environmental function of concern plays an important economic function (for example, air quality as an input in health, water quality as an input into agriculture, fish production, or human consumption, soil quality as an input to agricultural production,). For example, in the case of land degradation, indicators of achievable yield are more useful than indicators of soil depth. In this way, the indicators would speak directly to the problem of concern, and in most cases also give direct measures of project benefits (if the project is alleviating problems) or costs (if the project is causing them). The further the chosen indicator is from the economic endpoint, the more difficult it will be to evaluate the returns to the project.
Spatial and temporal coverage. Careful thought needs to be given to the appropriate spatial and temporal coverage of EPIs. Project activities might have an impact beyond the area in which the project is active. The affected area need not coincide with the national territory, however, so national-level measures may be inappropriate. (Where feasible, however, it is highly desirable that project-level indicators be comparable to national-level indicators.) There might also be lags before project effects are felt. Changes in the long-term status of biodiversity, for example, often only manifest themselves over time scales much longer than typical Bank projects.
Feasibility and cost. To be effective as an aid to decision-making, EPIs must be limited in number and should highlight essential factors in a concise manner. They must also be practical and realistic in terms of their costs of collection. This may lead to trade-offs between the information content of various indicators and the cost of collecting them. These trade-offs will obviously vary across technologies and depend heavily on institutional capacity. Certain indicators might be extremely simple or cheap to collect, but be inadequate for various reasons. The case of air pollution provides an example of the trade-offs that must often be made in selecting EPIs. Ideally, the project's impact on morbidity and mortality would be measured, since reducing them is generally the intended result. Morbidity and mortality themselves can be measured, but establishing a clear link between them and either ambient pollution levels (a state indicator) or any given source of emissions (a pressure indicator) remains extremely difficult, despite recent progress in this area (Ostro, 1994). In such cases, the only feasible solution in most cases is to fall back on indicators of ambient concentrations or, if the source has been established as contributing significantly to total pollution, of emissions.
Interpreting EPIs. Once an indicator has been selected and measured, it must still be interpreted. Emphasis has increasingly shifted towards performance indicators that measure changes relative to a goal established by environmental policy. Such an explicit reference to goals is important to put the project's impact in perspective. Once the project is underway, the emphasis is usually on variations in the indicator over time. A positive change in a state indicator or a diminution of a pressure indicator is usually considered an indication of success, as long as it can be shown that it is not the result of non-project factors or random effects (but this may require that baseline levels have been established for pre-project conditions, followed by measurements over extended periods to ascertain trends with confidence). The appropriate comparison, however, is generally not to the pre-project situation but to the counterfactual situation of what would have happened in the absence of the project. An increase in a pressure indicator could still be considered evidence of success if the pressure would have increased even faster in the absence of the project. In some cases, control groups can be used to measure conditions in areas not affected by the project; in others, statistical techniques need to be used to estimate what would have happened without the project.
There are a wide variety of airborne pollutants of concern from the point of view of health and environmental impacts. A number of site-specific studies have examined pollution risks and although results vary, there are some important consistent findings. Health problems have typically been associated with airborne particulates - measures of which include Total Suspended Particulates (TSP) and PM10 ( the more damaging smaller size particles) - and ambient lead. Damage to structures, forests, and agricultural crops tend to be primarily linked with SO2 and ground-level ozone.
Even though the ultimate objective of a project might be to mitigate damage to human health, monitoring such effects directly is extremely difficult because of substantial uncertainties over the exposure of different population groups to pollutants, their response to different levels of exposure, and the cumulative nature of damage. It is common, therefore, to fall back on monitoring indicators of ambient concentrations or of emissions (depending on the project's potential contribution to correcting the overall problem) to gauge a project's impact.
The most commonly used indicators of air pollution emissions and concentrations are listed in Table 1; these may need to be supplemented by additional EPIs depending on local conditions.
Industrial and agricultural chemicals, and organic pollutants from agro-based industries are a significant source of surface and ground water pollution. Acidification of surface waters from air pollution is a more recent phenomenon and is a threat to aquatic life.
Understanding of the impact of water quality on human health and aquatic life has improved enormously in recent years. Two broad measures of water quality have come to be widely used (see Table 1): measures of oxygen levels or demands in the water and heavy metals concentration. For example, a measure of pollutant concentrations could be considered as a pressure when measured in a stream that feeds into a lake, or as a state when measured in the water body fed by the stream. Used together, these indicators provide a rough but useful picture of the overall health of the water body or of the threats to it.
The procedures required for measurement of water quality indicators are problem-specific and are generally well understood. Sampling methods differ depending on whether the water body of interest is a lake or a stream, for example. Timing of measurements is often an issue, since concentrations can vary substantially as the flow varies; a given pressure may cause few problems when flow is at its peak but have a major impact at times of low flow.
Global Environmental Problems
Measuring the impact of projects on global environmental problems such as climate changes or damage to stratospheric ozone encounters significant scale problems. No single project is likely to have any measurable impact on these problems. Measuring the state of the problem, therefore, does not generally fall within the scope of project-level monitoring but determining the effect of a project on pressures is feasible.
Climate change. Climate change is linked to a number of important effects on the global life support system: sea level rise is just one of the most dramatic potential impacts; shifts in primary agricultural production areas are another. Although monitoring global climatic effects is impractical at the project level, emissions of greenhouse gasses (GHGs) give an indication of the pressures being generated. The most commonly used indicator in this area is some measure of carbon emissions (or other gases that contribute to global warming) or a measure of the percent reduction in carbon emissions from some base scenario. When multiple GHGs are involved, the global warming potential can be used as a weighting factor.
Stratospheric ozone. The ozone layer blocks ultra-violet radiation that is harmful to humans and all living resources. The degradation of the ozone layer is precipitated by ozone depleting substances (ODSs) such as chlorofluorocarbons (CFCs) and halons. Here too, monitoring global effects is impractical so work focuses on measuring changes in pressure resulting from project activities. The consumption and hence emissions of ODS can be used as a measure of the pressures being generated by economic agents. At the national level, production, net of exports and adding imports, can be taken as a proxy for the country's contribution to the problem. At the project level, the project's contribution to national production and consumption can be used as a proxy.