4 Compilation and processes used for the Net Position report 2005
This section of the report contains observations and recommendations that have arisen from our observation of the process used to compile the 2005 Net Position report projections.
We consider that the whole of Government approach used in compilation of the projections assessment is appropriate. A key advantage of this approach is that it allows those having technical expertise and sectoral knowledge in the respective sector areas to actively contribute to the projections work.
In previous years we recognise that significant institutional effort has been placed within Government departments on producing the annual national greenhouse gas inventory in order to meet the formal inventory reporting requirements laid down by the UNFCCC. In contrast, relatively few resources have been available for the compilation and reporting of the projection estimates. In all countries however (and not just New Zealand), there is now an increasing focus on projection estimates, both as the time to the first commitment period becomes closer and the implications of Kyoto Protocol targets become clearer. The importance (and profile) of reports concerned with projection estimates is therefore increasing.
For these reasons, New Zealand Government may wish to consider moving the process of compiling the projection estimates on to a more formal basis to help ensure the delivery of robust data. Amongst actions that might be considered are:
- The establishment of a steering group to oversee the projection compilation process. This might comprise representatives of all key Government Departments involved in the projection process and would help ensure a common understanding across Departments is achieved;
- Improved project planning and documentation. Use of formal project plans that reflect the increased importance of the projection calculation process as CP1 approaches. Ensure use of common datasets (see below);
- QA/QC plan for the projection compilation. The New Zealand national greenhouse gas inventory has a well established QA/QC process in place to ensure the robustness of the historical inventory data. A move towards establishing a formal QA/QC plan for the projections would help increase confidence in the numbers generated;
- Establishment of written documentation fully describing the methodologies employed in developing the projection and the inherent assumptions across all sectors, including a transparent quantification of the impact of Government policies.
A further recommendation arising from the review is that New Zealand consider aligning its definitions used in the projections work to those used by the UNFCCC. In the 2005 Net Position report, use of the terminologies 'most likely', 'pessimistic' and 'optimistic' for scenarios is somewhat unclear as to whether (and what) impacts arising from existing and future policy implementation are incorporated within scenarios or not. The use of the definitions prescribed by the UNFCCC (with measures, with additional measures etc) would provide the advantage of having clearly understood scenario definitions.
A related issue is the treatment of uncertainty. Uncertainty is treated slightly differently in each of the sectors considered. In most cases there is some consideration of the uncertainty caused by unpredictability (e.g. uncertainty in forecasts of key drivers such as GDP, fuel prices), to derive the pessimistic/optimistic scenarios. However in some sectors this is also combined with estimates of value uncertainty. It might be useful to consider the contributions to overall uncertainty from structural uncertainty and from structural and value uncertainty separately to assess their relative importance.
We also note that work is currently underway compiling projection estimates for New Zealand's Fourth National Communication (where the UNFCCC projection terminologies of 'with measures' and 'with additional measures' must be used). As these definitions will differ somewhat from those used in the 2005 Net Position report (where terminologies of 'most-likely' and 'optimistic' scenarios were used), we recommend that careful consideration is given as to how the (inevitable) changes from the numbers presented in the Net Position report are presented. The definitions behind the different series should be clearly explained.
We recognise that good efforts have been made to ensure the use of common datasets and assumptions across different sectors. For example, the NZ Treasury GDP forecasts have been used across several sectors (energy, agriculture etc). However, we have identified two instances where despite the efforts made, different datasets ands assumptions have been used:
- use of different population statistics for the energy modelling and waste sector calculations;
- use of different assumptions of deforestation rates for land converted to pasture for the LULUCF and agriculture sectors.
It is important that projection estimates are consistent across different sectors and share the same input data where appropriate.
A final issue we have noted with respect to the existing compilation process is the risk of double counting of policy effects. We have already noted in Section 3 our observation that there is a risk that emission savings attributable to NEECS (energy audits and loans and grants) may overlap with those arising from the SME/EIB policies. A related situation concerns a discrepancy noted concerning PREs. In this instance the liabilities arising from PREs (and which are used in the compilation of the overall net position) are not consistent with the internal results of SADEM in relation to the assumed success rate for projects going forward, meaning the assumed liabilities were greater than they should have been (7.5 Mt CO2 eq vs 6.3 Mt CO2 eq). We therefore recommend that for future projection forecasts care is taken to ensure internal consistency in these areas.
