Projections in hindsight - An assessment of past emission projections reported by Member States under EU air pollution and GHG legislation


Courtesy of European Environment Agency (EEA)

Over the past decade, European Union (EU) Member States were required to regularly compile and report official estimates of their projected (future) emissions for the year 2010 under two complementary pieces of legislation:

  • for greenhouse gases (GHGs), the Monitoring Mechanism Decision (MMD), i.e. Decision No 280/2004/EC of the European Parliament and of the Council of 11 February 2004 concerning a mechanism for monitoring Community greenhouse gas emissions and for implementing the Kyoto Protocol;
  • for air pollutants (specifically four atmospheric pollutants responsible for acidification, eutrophication and ground-level ozone pollution), the National Emission Ceilings (NEC) Directive, i.e. Directive 2001/81/EC of the European Parliament and of the Council of 23 October 2001 on national emission ceilings for certain atmospheric pollutants.

The mandatory reporting of projections plays an important part in the policy process: it is designed to provide Member States, the European Commission and other stakeholders with a reliable early indication of the expected trend of future emissions, and thus with information on whether implementation of national and EU-wide policies and measures is helping countries meet their emission commitments.

The year 2010 was the target year for which the NEC Directive set upper limits (ceilings) for each Member State for total emissions of certain air pollutants. Under the MMD, Member States were required to report future estimates of GHGs for the years 2005, 2010, 2015 and 2020. Actual emission inventory data for the year 2010 is now available (EEA, 2014c) and it is possible to compare reported emission projections with historic emission inventory data for 2010 (1). The current assessment compares the successively reported past projections for 2010 with the latest emissions inventory data for that year, as officially reported by Member States and made available to the EEA.

In performing this comparison, two key questions are addressed:

  • How do the past emission projections for 2010 reported by Member States over the last decade compare with the actual emission inventory data reported for that year?
  • To what extent have national projections, as they were compiled and reported in the past, proven to be 'fit for purpose', in the sense of adequately addressing their intended purposes as specified in the MMD and NEC Directive?

Based on this assessment and on observations made in the process of monitoring, reporting and interpreting emission projections over the last decade, several shortcomings and quality issues have been identified. An overview of these shortcomings and quality issues is presented in this report, together with recommendations on how these might be overcome in future reporting under a revised NEC Directive and the new GHG Monitoring Mechanism Regulation (MMR) (i.e. Regulation (EU) No 525/2013 of the European Parliament and of the Council of 21 May 2013 on a mechanism for monitoring and reporting greenhouse gas emissions and for reporting other information at national and Union level relevant to climate change and repealing Decision No 280/2004/EC). It is recognised that overall, the reported national projections lack detailed information on methodologies, consistent information on exogenous key parameters and explanations on the link between policies and measures and projections. Hence, this report does not review the underpinning methodologies or models used by Member States to develop their projections, nor does it provide an in-depth review of the past reported projections scenarios.

The aggregated, average differences between reported projection, 'with existing measures' (WEM) scenarios, and 2010 emission inventory data for the four air pollutants addressed in the NEC Directive and for total GHGs reported under the MMD. It is clear there are systematic differences between projections reported over past years, and the emission inventory data for 2010 eventually produced. For both air pollutants and GHGs, there was a common trend: the reported projections overestimated the eventual 2010 emissions, albeit to a greater extent for air pollutants than for GHG projections. Even in years immediately preceding 2010, when good agreement between reported projections and final emissions inventory data might be expected, there were several instances for which the difference exceeded 100% in certain Member States.

On a more positive note, however, the average aggregated underestimation or overestimation did decrease, which means that projections mostly converge over time towards the value of the actual 2010 emissions. A successively reported WEM scenario for a specific year should ideally converge to the emission inventory value ultimately reported, since reduction effects of adopted and implemented policies and measures become increasingly visible in the trend of the reported projections.

The reliability of the reported projections, in terms of their ability to inform distance-to-target assessments, has therefore improved over the period, even if substantial differences still occurred often in the years close to 2010 for certain Member States.

A number of observations were made concerning the reported projections. For GHGs, it was assumed that there were significant quality issues in Member States' emission projections frameworks if the year 2010 was overestimated or underestimated by more than the country-specific uncertainty of the respective emission inventory for two submissions close to 2010, i.e. 2009 and 2011. These countries were Lithuania, Portugal and Romania. Bulgaria did not report projections in 2009, but exceeded the country-specific inventory uncertainty in its 2011 submission. Poland did not report projections in 2009 and 2011.

For air pollutants, in certain years a third of the Member States have simply reported the value of their 2010 emission ceiling as their 'with measures' scenario. This is not in line with the good practice requirements for the compilation and reporting of projections as laid out in the EMEP/EEA air pollutant emission inventory guidebook 2013 (EMEP/EEA, 2013, and earlier editions). For sulphur dioxide, a difference of more than 200% was observed for one or more years in the cases of Denmark, Hungary, Latvia, Lithuania and Portugal. In 2009, close to the target year 2010, three Member States reported overestimations of more than 100% (Hungary, Portugal and Romania). For NOX, Bulgaria, Lithuania and Romania showed differences higher than 50% at least in one of the later submission years (2008, 2009 and 2010). Bulgaria, Malta and Slovakia showed overestimates above 50% in at least one of the submission years of 2008, 2009 and 2010 for non‑methane volatile organic compounds (NMVOC). For NH3 and the same submission years, Bulgaria and Lithuania overestimated their projected emissions for the year 2010 by more than 100%.

A significant transparency issue noted in this assessment was that it was not possible, on the basis of the information reported by Member States under either the NEC Directive or GHG MMD legislation, to identify the reasons for past changes in the reported projections. In particular, more consistent and complete information on national policies and measures and the links between measures and projections would have been needed to identify reasons related to policies and measures, which are ultimately the main driving factors of interest in the WEM, WAM and WOM set of scenarios when it comes to monitoring Member States' progress in meeting targets.

Reasons for differences between projected scenarios and the inventory value may include:

  • policies introduced in addition to those planned, and not included in earlier WEM or WAM scenarios (e.g. change in climate strategy due to changes in government, for instance);
  • policies included in the scenarios that fail on the ground;
  • overestimated or underestimated reduction effects and/or penetration rates of policies and measures included in projections;
  • rebound effects or interactions between policies and measures that are overestimated, underestimated or not considered in scenarios;
  • changing trends or misinterpreted trend expectations for key parameters, or poor quality of statistical inputs (e.g. concerning the economy, energy, fuel-prices, traffic forecasts);
  • uncertainties intrinsic to the modelling used to project model parameters;
  • outdated emission projections;
  • unforeseeable events that cause differences between rates of economic activities compared to what was originally projected, e.g. the economic recession in recent years, or political decisions affecting fuel use such as the closure of nuclear plants;
  • errors in the reported data.

To help improve the fitness for purpose of projections information from Member States, the report identifies a number of actions addressing both GHG projections and air pollutant projections.

  • Improving overall quality of national projections. The significant differences between projected values for 2010 in the submission years 2009 and 2011 for GHGs, and in the submission years 2008, 2009 and 2010 for air pollutants, indicate that some countries are still facing major problems in developing reliable projections. This means that targeted capacity‑building efforts for the development of GHG and air pollutant projections should be further continued. Furthermore, Member States must also allocate sufficient resources in order to ensure a functioning national system for the development of emission projections.
  • Improving the implementation of existing reporting requirements for WEM and projections information. Despite formal requirements that Member States regularly report projections data, in practice not all Member States reported data in each of the years for which reporting was required. While the completeness did improve over the time period, even by 2010, various Member States had not reported the mandatory background parameters and indicators underpinning the reported projections, despite it being mandatory to do so. A renewed focus on enforcing the relevant reporting requirements under the NEC Directive and the new MMR would greatly support progress assessments.
  • Improving the implementation of existing reporting requirements for WAM and WOM scenarios. If emission projections are to properly inform policymakers of progress being made in achieving policy targets, the projected trends of a set of scenarios should be taken into consideration together with background information on how implemented and planned policies and measures have been integrated in the different scenarios. A WOM and a WAM scenario should complement the interpretation of the projected progress to target assessment of a WEM scenario, as differences in these scenarios provide insight into the sum of policy effects of either implemented measures (WEM–WOM) or additional measures (WAM–WEM). A renewed focus on Member States making such information available is essential in order to appropriately inform distance-to-target assessments.
  • Increasing the transparency of reported projections and the link to policies and measures. There remains a very wide variety in the consistency and level of detail of information reported by Member States, especially when it comes to quantification of expected policy effects. Clearer guidance for Member States in this area would result in a deeper understanding of the progress being made to policy targets. Most importantly, information concerning the manner and degree to which national and EU‑wide policies and measures are incorporated into the national projections is generally poor, with detailed information on the timing and expected implementation rates of different policies generally not reported by Member States.
  • Exercising caution in interpreting reported projections. Over the past decade, projections reported by Member States have generally been overestimated for different reasons, even in years immediately preceding 2010. However, the reliability of the reported projections, in terms of their ability to inform distance-to-target assessments, has improved over the period. Policymakers should remain aware of the high degree of uncertainty inherent in projections, especially in years distant from a target year; they should insist on the provision of information regarding uncertainty of methodologies and models, e.g. sensitivity analyses together with projection results, as explicitly encouraged in the MMD and again promoted in the new MMR. A renewed focus on Member States making available such information is essential.
  • Improving reporting to better understand the uncertainties associated with projection estimates. Reporting requirements under the NEC Directive did not inherently require projections to address uncertainty, focusing instead on the provision of a central WM scenario based upon existing and future policy measures and also reporting of key projection parameters. For GHG projections, Member States were required to provide sensitivity analyses of their projections, and were encouraged to develop and report a high, central and low scenario. If policymakers consider it desirable to better understand the likely impact upon future emissions of 'unforeseeable' events (e.g. a significant reduction of gross domestic product (GDP) which might occur in a period of unexpected economic recession), implementing measures to enhance the reporting of a meaningful sensitivity analysis would be beneficial.
  • Promoting complementary methods for assessing progress to policy targets. The assessment presented in this report shows that although the reliability of projection estimates improved over time, in a number of instances the reported projections provided a significantly overestimated picture of the emissions finally reported in 2010. The uncertainty intrinsic to projections can hamper the robustness of (additional) policy decisions being considered on the basis of these values alone, particularly if based upon a single scenario reported in years in a relatively short time period preceding a target year or period. One illustration of a complementary system for monitoring 'distance‑to‑target', compared to a system based solely upon projected values, is using analyses based upon current emission inventory values relative to a target trajectory.

Projections are important for tracking progress towards policy targets, and especially for longer term evaluation of progress, such as for 2030 or 2050, projections will remain the main tool. Target trajectories will not be universally rolled out across all policies, but it is recommended that the evaluation process place more emphasis on the years preceding a target year, and apply an approach combining current and projected progress. Policymakers need to become aware of the uncertainties inherent in projections, but countries can do more to make their projections more robust and transparent.

Customer comments

No comments were found for Projections in hindsight - An assessment of past emission projections reported by Member States under EU air pollution and GHG legislation. Be the first to comment!