4 Uncertainties

4.1 Introduction

The most likely estimate of the net position is calculated as the sum of the sector emissions less the net removal by forestry. The calculation of the upper and lower bound estimates of the net position from the sector upper bound and lower bound estimates is more difficult because it is highly unlikely that all pessimistic or all optimistic situations will occur together. In 2005 and 2006, a Monte Carlo simulation was carried out to model the combined uncertainty. The Monte Carlo simulation is a repeated sampling technique in which a probability distribution is assumed for each of the sector emissions estimates and the net removal by forestry. Repeated samples are drawn from each distribution and the net position calculated each time. The resulting estimates of the net position have a modelled probability distribution from which a 95% confidence interval for the upper and lower bounds may be obtained. There is a debate as to whether the Monte Carlo method is a suitable tool for modelling the uncertainty in the emissions projections for New Zealand because of the limited number of modelled projections, and because the overall uncertainty is dominated by the large uncertainty in the estimates of net removal by land use, land-use change and forestry. No attempt has been made in the 2007 assessment to combine the uncertainties in the sector emissions estimates.

This section provides guidance on how best to treat this uncertainty. In section 4.2, consideration is given to the limitations of the Monte Carlo approach taken in 2005 and 2006 for combining sector estimates of uncertainties. A revised approach is recommended.

The combination of the uncertainties in the overall net position relies on the estimates of the uncertainties in each of the emission sectors, and in the estimation of the net removal by forestry. The uncertainty is dominated by the uncertainty in the land use, land use change and forestry sector. The method of combining the components of the uncertainty in this sector is considered in section 4.3.

Section 4.4 provides a summary of our recommendations on combining uncertainties. Section 4.5 looks in more detail at estimating uncertainties in sectoral emissions estimates, and section 4.6, uncertainty in LULUCF emissions estimates. Our overall recommendations on uncertainty analysis are given in section 4.7.

4.2 The combination of uncertainty estimates

In an ideal world, the net position would be calculated by a single integrated model. All the inter-relationships and correlations between the input variables would be known accurately and a Monte Carlo simulation could be carried out by repeatedly sampling the input variables from their joint probability distribution. In reality, the expertise required to develop and maintain the models for each of the emission sectors lies within various bodies within and outside government. Projections of emissions from the agricultural sector are based on modelling by the Ministry of Agriculture and Forestry; the Ministry of Economic Development provides estimates of annual emissions from the energy, transport and industrial process sectors; the Ministry for the Environment provides estimates of emissions from the waste sector; projected removals from land use, land use change and the forestry sector are based on data and assumptions from the Ministry of Agriculture and Forestry and the Ministry for the Environment; carbon modelling for the forestry sector was undertaken by Ensis.

4.3 Common economic assumptions

The projections made for the various sectors depend to some extent on the general economic conditions. In our review of the 2005 assessment, AEA recommended that projections for each sector were based on a common set of economic assumptions. This recommendation was adopted in preparing the May 2006 projections. The Treasury and the Ministry of Economic Development prepared key assumptions for the most likely scenario for the 2006 assessment (Table 4). The draft 2007 assessment report does not present updated projections for the economic indicators.

Table 4: Most likely economic scenario

March year ending	Economic growth (GDP), % per annum	Exchange rate NZ$/US$	Population (000)	Oil prices, US$ /bbl
2005	3.8	0.69	4093	60
2006	2.9	0.60	4130	60
2007	1.7	0.60	4166	60
2008	2.5	0.56	4204	60
2009	3.8	0.55	4240	60
2010	3.1	0.55	4275	60
2011	2.5	0.55	4312	60
2012	2.5	0.55	4350	60

It is not clear whether all departments have prepared common assumptions for the upper and lower bound scenarios.

In defining upper and lower bounds for the economic assumptions, it is necessary to consider the possibility that significant correlation exists between the growth in GDP, the exchange rate and the price of oil. It is also necessary to consider the extent to which these economic indicators are correlated between years. For example, it is possible that GDP would grow at rates above the most likely projection in one year, but increasingly unlikely that growth above projection would occur year on year into the future. On the other hand, a high oil price in one year may be followed by high oil price in subsequent years.

In an ideal world, the Treasury would produce large numbers of economic scenarios conforming to the joint probability distribution for economic growth, exchange rate, population and oil prices. These scenarios could then be used as inputs to the sector models. In reality, it may be only practical to limit the number of economic scenarios to the most likely, and the upper and lower bound scenarios.

4.4 Combining uncertainties

The modelling studies prepared by the relevant experts provide upper and lower bound estimates of the carbon emissions or net removal from various sectors. The IPCC Good Practice Guidance Annex on Uncertainty describes two basic approaches to the combination of emission uncertainties:

Tier 1, based on simple linear analysis of variance.
Tier 2, based on Monte Carlo simulation.

The Tier 1 approach is generally applicable in the case where the component uncertainties are relatively small and there is no correlation between the component uncertainties. These requirements are not met because there are relatively large uncertainties in the emissions from the agricultural sector and in the net removal from forestry and there may be significant correlation between sectors. It is therefore most appropriate to use the Monte Carlo simulation for combining uncertainties.

Table 5 shows the emissions and removals reported in the draft 2007 assessment of the net position.

The New Zealand Government’s current policy is to cap the Crown’s deforestation liability for pre-1990 forests at 21.0 million tonnes of carbon dioxide. A deforestation survey undertaken in 2006 indicated that deforestation is likely to exceed the 21.0 million tonne cap in the absence of policy interventions under current market conditions. The 2006 deforestation intention survey indicated that forest owners currently intend to deforest about 50,000 hectares during the first commitment period of the Kyoto Protocol. This area would generate deforestation emissions of approximately 41.0 million tonnes carbon dioxide. There are clearly two distinct scenarios possible, with potentially very different results. If the Government applies effective measures to limit deforestation to the cap, then the deforestation emissions will be 21.0 million tonnes carbon dioxide; if no measures are put in place the emissions will be approximately 41.0 million tonnes carbon dioxide. It is not appropriate to include this policy uncertainty in the analysis of the overall uncertainty; rather the two scenarios should be considered separately. For this review, we will consider that effective measures to control the emissions to the level of the cap are put in place.

Table 5: Emissions and removals reported in the draft 2007 assessment

	Million tonnes of carbon dioxide equivalent
	Upper scenario	Most likely scenario	Lower scenario
Emissions
Energy (excluding transport)	-103.0	-92.8	-86.1
Transport	-84.7	-80.1	-76.7
Industrial processes	-22.3	-22.2	-22.1
Solvent and other product use	-0.3	-0.3	-0.3
Agriculture	-228.3	-203.1	-180.0
Waste	-7.3	-7.0	-6.7
Assigned amount units
Assigned	309.5	309.5	309.5
Allocated to projects	-7.5	-7.5	-7.5
Removals based on afforestation	57	79	119.3
Deforestation emissions	-41.0 (-21.0 with cap)	-21.0 (cap)	-21.0 (cap)

An important consideration is whether the emissions from each of the emissions sectors can be considered to be independent of each other. A Monte Carlo simulation was carried out for the 2006 assessment based on the assumption that that the emission uncertainties in each sector were independent of the other sectors.

The analysis indicated a 95% confidence range in the projected balance of emission units of –76.1 to +1.4 million tonnes of carbon dioxide. Applying a similar analysis to the data presented in the 2007 draft report gives a projected range of –82 to +3 million tonnes carbon dioxide. The analysis assumed that the probability distribution of emission for each sector corresponded to a triangular distribution with mode equal to the most likely value and 5% of the distribution lying outside the upper and lower scenario bounds. Ten thousand simulations were carried out.

The emissions from the energy sectors, including transport, are all dependent on the level of economic activity. The emissions in these sectors are therefore likely to be substantially correlated and it may not be appropriate to consider them to be independent. The Monte Carlo analysis was repeated assuming that the emissions from the energy, transport, industrial process, solvent and waste sectors were not independent. The upper bound range of emissions for the combined sector was calculated as the sum of the upper bound emissions from the sectors. Similarly, the lower bound was calculated as the sum of the lower bound emissions. This analysis gave a 95% confidence range in the projected balance of emission units of –83 to +3 million tonnes of carbon dioxide. Comparing the results with those obtained assuming independence between the energy sectors, it is clear that the assumption of independence for these sectors does not affect the results significantly.

Emissions from the agricultural sector are also dependent on the economic conditions. However, the correlation between the agricultural and energy sector emissions may not be strong and may in some circumstances be negative. For example, conditions giving rise to a high rate of economic growth may also lead to a reduction in the demand for agricultural exports. The assumption of negative correlation between the energy and agriculture sectors reduces the predicted 95% confidence range in the projected balance of emission units to –73 to –5 million tonnes of carbon dioxide. It is clearly important to take account of how the agricultural and energy sector emissions are correlated and it is recommended that future work should establish this as set out in sections 4.5, 4.6 and 4.7.

4.5 Sector emissions

In the assessments carried out to-date, the upper bound emission for each of the emission sectors was calculated assuming worst-case values for each of the model input parameters. The lower bound emission was calculated assuming best-case values for each of the model input parameters. Thus the upper emissions scenario for agricultural emissions combined the upper 95th percentile projection for animal numbers, methane emissions per head, nitrogen output per head and nitrogen fertiliser use. It seems unlikely that all the optimistic or pessimistic options would occur together.

For each component within each sector, the emission is calculated as the product of the activity level and the appropriate emission factor, both of which are subject to uncertainty. The uncertainty in the activity levels is comprised of the uncertainty associated with the overall levels of economic activity and the uncertainty associated with other factors. Each of the sectors is affected to a greater or lesser extent by economic conditions and so there is the potential for correlation to occur between the emissions. As shown above, the overall uncertainty in the net position is affected by the extent to which the agricultural sector and energy sector emissions are correlated. It is thus important to distinguish between these two sources of uncertainty (economic and non-economic). It is therefore recommended that sector model runs are carried out for the most likely, pessimistic and optimistic economic scenarios with all other parameters set at their most likely values. This will provide an estimate of the uncertainty in the sector emissions resulting from the uncertainty in the overall economic factors.

The remaining task for each sector is then to assess the uncertainty associated with the non-economic factors. Pessimistic, most likely and optimistic assessments can be made for the most likely economic scenario based on the most optimistic or pessimistic non-economic input values.

The results from the different sectors can then be combined by Monte Carlo simulation of the sum of three independent distributions, each with triangular probability function:

A triangular distribution with mode equal to the sum of the most likely energy and agriculture emissions for the most likely economic scenario; upper bound equal to the sum of the most likely energy and agriculture emissions for the pessimistic economic scenario; and lower bound equal to the sum of the most likely energy and agriculture emissions for the optimistic economic scenario.
A triangular distribution with mode equal to zero; upper bound equal to the difference between the upper bound and most likely estimates of energy emissions for the most likely economic scenario; and lower bound equal to the difference between the lower bound and most likely estimates of energy emissions for the most likely economic scenario.
A triangular distribution with mode equal to zero; upper bound equal to the difference between the upper bound and most likely estimates of agriculture emissions for the most likely economic scenario; and lower bound equal to the difference between the lower bound and most likely estimates of agriculture emissions for the most likely economic scenario.

The procedure is illustrated by example. Suppose that model results are obtained for the energy and agriculture sectors for high, most likely and low economic growth rates as shown in Table 6. The numbers are hypothetical and do not represent the actual situation.

Table 6: Hypothetical model result for combining uncertainties

Scenario	Million tonnes of carbon dioxide
Scenario	Energy	Agriculture
High economic growth scenario, most likely non-economic factors	215	197
Most likely economic growth scenario, most likely non-economic factors	202	203
Low economic growth scenario, most likely non-economic factors	194	205
Most likely economic growth scenario, pessimistic non-economic factors	204	210
Most likely economic growth scenario, optimistic non-economic factors	200	197

The first distribution, representing the range of the total (energy plus agriculture) emissions for the most likely non-economic factors, has mode 405 (= 202 + 203), upper bound 412 (= 215 + 197) and lower bound 399 (= 194 + 205). The second distribution, representing the uncertainty associated with non-economic factors in the energy sector, has mode 0; upper bound 2 (= 204 – 202) and lower bound –2 (= 200 – 202). The third distribution, representing the uncertainty associated with non-economic factors in the agriculture sector, has mode 0; upper bound 7 (= 210 – 203) and lower bound –6 (= 197 – 203).

The procedure is then to carry out a Monte Carlo simulation to calculate the distribution of the sum of these three distributions.

4.6 Land use, land use change and forestry emissions (LULUCF)

The estimates of the net carbon removal resulting from land use, land use change and forestry (LULUCF) introduce the largest element of uncertainty into the overall estimate of the net position.

The Ministry of Agriculture and Forestry has run three simulations using all of the high emissions, most likely emissions and low emissions assumptions respectively to produce combined model results. The Ministry has also investigated the effect of changing the main model assumptions individually to investigate the sensitivity of the emissions to each of the main assumptions. Table 7 shows the results of these calculations presented in the draft 2007 report.

Table 7: Net removal from land use, land use change and forestry

	Million tonnes of carbon dioxide equivalent
	Upper scenario	Most likely scenario	Lower scenario
Removals based on afforestation
Kyoto planted forest carbon dioxide emissions	96.8	96.8	96.8
Future afforestation	0	0.9	1.9
Adjustment in the area of Kyoto forest planted between 1990 and 2006	-4.8	0	4.8
Kyoto forest growth rate adjustment	-9.8	0	28.4
Soil carbon change with afforestation	-11.3	-3	0
Ineligible afforestation	-20.5	-15.7	-7.8
Total removals from afforestation	57	79	119.3
Deforestation emissions	-41.0 (-21.0 with cap)	-21.0 (cap)	-21.0 (cap)

The combined model results account for the interrelationships between adjustment factors (growth rates, new planting, soil carbon changes, over planting and scrub clearance during site preparation). The draft report states that the removals attributed to each factor are not additive because some factors are correlated. As an example, the report indicates that the soil carbon change is reduced as the result of ineligible planting because the total eligible area is reduced. This non-linearity results in a reduction in the range between the upper and lower bound estimates.

It seems possible that a further reduction in the estimate of the uncertainty of removal by afforestation is possible. It is unlikely that the uncertainty in the area of Kyoto forest planted between 1990 and 2006, the Kyoto forest growth rate per hectare, the soil carbon change per hectare, and the ineligible afforestation could be strongly correlated. It is unlikely that all the high emissions conditions would occur at the same time. It is not possible to quantify by how much the uncertainty range can be reduced exactly because we do not have sufficient details of the combined model. However, a preliminary assessment can be made if the non-linearities in the combined model are ignored. We have carried out a Monte Carlo simulation of the removals based on afforestation. Each of the ranges shown in Table 6 was represented by a triangular probability distribution with mode equal to the most likely value, the high emissions value equal to the 97.5th percentile of the distribution and the low emissions value equal to the 2.5th percentile. The 95th percentile range of the predicted values was 69 to 109 million tonnes carbon dioxide equivalent.

The reduction in the uncertainty in the LULUCF net removal has a substantial effect on the overall uncertainty in the net position. We have combined the Monte Carlo above with the simulation of the net position assuming correlation between the energy sectors and waste. This analysis gave a 95% confidence range in the projected balance of emission units of –73 to –4 million tonnes of carbon dioxide. This range is substantially smaller than the equivalent range of –83 to +3 million tonnes of carbon dioxide calculated in section 2.

4.7 Recommendations

The following recommendations are made on the uncertainty analysis:

That all departments should develop a range of consistent economic scenarios including the most likely, optimistic and pessimistic scenarios. In developing these consistent scenarios, the potential correlation between economic indicators should be taken into account.
Modellers in the Ministry of Economic Development and the Ministry of Agriculture and Forestry should develop the most likely estimates of the emissions for the energy and agricultural sectors for each of the most likely, optimistic and pessimistic scenarios of economic factors. In addition, the uncertainty in the energy and agricultural emissions associated with non-economic factors should be assessed.
The estimates of the uncertainty from each of the sectors can be combined by Monte Carlo simulation, provided that the potential correlation between sectors resulting from economic factors is taken into account. We have suggested a simple procedure.
The New Zealand Government’s current policy is to cap the Crown’s deforestation liability for pre-1990 forests at 21.0 million tonnes of carbon dioxide. If this policy is not implemented effectively, deforestation emissions are expected to rise to 41.0 million tonnes. It is not appropriate to include this policy uncertainty in the analysis of the overall uncertainty; rather the two scenarios should be considered separately.
The uncertainties in the LULUCF emissions make the biggest contribution to the uncertainty in the net position. Currently upper and lower bound estimates are calculated assuming all high and all low values respectively for each of the factors contributing to the net removal. Monte Carlo simulation within this sector has the potential to reduce the uncertainty in the net removal and the overall net position substantially.

Previous|Next