Review of New Zealand's 2007 Net Position Report

• Executive Summary
• 1 Introduction
• 2 General Review
• 3 Sector Reviews
• 4 Uncertainties
• 5 Conclusions and Recommendations
• 6 References

2 General Review

The overall approach taken in compiling the net position report was reviewed in 2005. In general it was considered that the government approach used in compilation of the projections assessment is appropriate, allowing those having technical expertise and sectoral knowledge to actively contribute to the projections work. It was, however, suggested that as the first commitment period approaches and the importance of reports concerned with projection estimates increases, the New Zealand Government might wish to consider moving the process of compiling the projection estimates on to a more formal basis to help ensure the delivery of robust data. A number of actions were suggested. As the commitment period moves ever nearer, then the development of a more formal basis for the projections and compilation process becomes increasingly important. The recommendations made in 2005 are discussed below, together with our assessment of progress towards their implementation, and suggestions for further improvements which could be made.

2.1 Improving planning and quality assurance/ quality control (QA/QC)

Three actions were suggested to improve planning and QA/QC:

• establish a steering group to oversee the projection compilation process

• improve project planning and documentation

• introduce and implement a formal QA/QC plan for the projection compilation.

A steering group has been established and includes representatives from the Ministry for the Environment, the Ministry of Economic Development, the Ministry of Agriculture and Forestry, Department of the Prime Minister and Cabinet and the Treasury. They met in September 2006 to review the 2006 compilation process, and in January, March and April 2007 to discuss the compilation of the 2007 net position report. A smaller group of technical officers also met more frequently to progress the compilation of the report.

Minutes from the steering group meeting and discussion with Ministry for the Environment officials suggest that project planning for compilation of the report has improved (for example, departments are given guidance on writing up the projections and on treatment of uncertainty). In terms of QA/QC, chief executives are now asked to sign a letter stating the figures provided are, in the opinion of the relevant department, the best available estimate. This encourages internal review of the reports before submission of final reports to the Ministry for the Environment.

The Ministry for the Environment informed us that it then reviews the sectoral reports and discusses any issues with the contributing departments. Once the net position review report has been drafted, then this is reviewed by the departments and by senior officials at the Ministry for the Environment. In addition, a member of the inventory team is asked to perform numerical checks.

We would suggest that the Ministry for the Environment continue to build on work done to improve project planning and QA/QC by ensuring that the processes being put in place are documented, monitored and reviewed. More specifically:

• All those providing projections for compilation in the net position report should prepare a written QA/QC document covering the preparation of projections. The letter from the chief executives accompanying submissions could be amended to state that the projections have been prepared according to this QA/QC document. The Ministry for the Environment should also produce a written project plan for compilation of the projections and a QA/QC document covering the written projections. From discussions with the officials in New Zealand, it appears that there is already significant checking and review of the projections, but it is important that this process is recorded so that there is a written audit trail.

• A written plan to improve the quality of projections should be produced. This should set out a timetable for specific actions over the next two to three years to continue to improve the quality of the projections. Actions should be allocated to named individuals, progress against the plan reviewed periodically, and further actions added as necessary.

• Actions should focus on sectors/sub-sectors where data or modelling is judged to be weak, and should be prioritised according to how significant the associated emissions are judged to be in the projections.

2.2 Improving internal consistency of projections

The 2005 review recommended that the internal consistency of the projections be improved by reinvigorating efforts to ensure the use of common datasets across all sectors, and by ensuring consistency in assumptions common to different sectors.

The Ministry for the Environment has put considerable effort into ensuring that this happens and that common datasets are used (for example, population, gross domestic product (GDP) growth and fuel prices) and common assumptions are made wherever possible. For example the projections of dairy herds made by the Ministry of Agriculture and Forestry are now used as a basis for estimating the energy requirements of the dairy processing industry.

It was specifically noted in the previous review that different assumptions of deforestation rates for land converted to pasture had been used in the LULUCF and agriculture sector projections. A post-model adjustment had been made to the Pastoral Supply Response model to allow for the conversion of grazing land to forestry and deforestation (with land going to dairy farming), but the assumptions made were not consistent with those made in the LULUCF sector, which led to the above recommendation. We were informed that in the 2007 net position report there was no post model adjustment to account for afforestation and deforestation, and that information from experts in the forestry sector suggest that land conversion only marginally impacts the pastoral sector emissions. Therefore any land use changes in the forecast period are assumed to be captured by the residuals (errors) of the forecasts. A simple, approximate calculation of the emissions involved confirms that annual emissions from agricultural use of deforested land would be of the order of 1% of those arising from deforestation,² supporting the assertion that the emissions from deforestation dwarf those which could arise from additional livestock emissions if the land was used for grazing.

2.3 Treatment of uncertainty

The review of the 2005 net position report recommended improving the transparency of the uncertainty calculations by clearly defining the basis for calculating the uncertainty ranges and using a consistent approach to quantifying uncertainties across sectors in terms of deciding what to quantify; that is, the uncertainty caused by future unpredictability (for example, uncertainty in the forecasts of key drivers) or value uncertainty (such as, uncertainty in data values).

The Ministry for the Environment has attempted to improve the treatment of uncertainty and has provided guidance to contributing departments on how uncertainty should be dealt with. The departments have also expanded the number of causes of uncertainty that they are examining. For example, the Ministry of Agriculture and Forestry has improved its treatment of uncertainty by examining how variations in commodity prices affect the modelled estimates of livestock numbers, and the Ministry for Economic Development has explored the effect that a number of discrete events (for example, Methanex not operated) might have on the projections, as well as, for example, the impact of high and low population growth. However, there is still no differentiation between uncertainty introduced by future unpredictability, value uncertainty and uncertainty caused by discrete events (for example the lifetime of Methanex plant), and/or assumptions about the outcome of policies. The treatment of uncertainty is discussed in more detail in section 4 of this report.

2.4 Impact of policies

General recommendations made in 2005 regarding the analysis of policy impacts were:

• Improve the documentation available concerning the basis of the attributed emission savings anticipated from climate change policies (particularly for the optimistic and pessimistic estimates).

• Further integrate the analysis of policy impacts to ensure that there is no double-counting of savings; with regard to this recommendation particular concerns identified were that emission savings attributable to the National Energy Efficiency and Conservation Strategy (energy audits and loans and grants) may overlap with those arising from the Small and Medium-sized Enterprises/Energy Intensive Businesses (SME/EIB) policies.

Better documentation is now included with the net position report, but it is felt generally that a transparent quantification of the impact of Government policies is still not provided in all cases. In some cases references are not provided for documentation setting out the assessment of impacts.

The 2007 draft net position report states that emissions savings from the National Energy Efficiency and Conservation Strategy and the Energy Intensive Businesses programme are explicitly represented in the SADEM energy model projection. As this current review is not as detailed as the 2005 review, it has not been possible to check whether this has resolved potential double counting of savings.

The 2005 review also recommended that a discrepancy concerning ‘Projects to Reduce Emissions’ (PREs) was resolved. The liabilities arising from PREs (which are used in the compilation of the overall net position and are discussed in section 3.1) were not consistent with the internal results of the Supply and Demand Energy Model (SADEM) in relation to the assumed success rate for projects going forward. However, most of the PREs are generation projects in the electricity sector, and so they are now modelled in the electricity sector’s Generation Expansion Model (GEM) rather than the SADEM energy model. In consultation with the Electricity Commission, only those projects which are sufficiently advanced that they are likely to be installed over the commitment period are included. The assessment of PRE liabilities is made by the Ministry for the Environment which estimates the number of project units likely to be awarded based on contact with the PREs. It is not clear whether the assumptions in the GEM are cross-checked with the Ministry for the Environment assessment.

---

---

Previous|Next