Skip to main content.

2 Heating Source Evaluation Methodology and Model

Methodology overview

The evaluation methodology that has been developed is based on the idea that to determine how well different heating sources will meet a householder's needs, it is necessary to first determine the:

  • heating requirements of the household
  • characteristics of the heaters being considered
  • performance of the heaters when attempting to meet the heating requirements
  • relative performance of the heaters against specified rating criteria.

An overview of the process used to derive the final ratings, and the factors influencing the ratings, is illustrated in Figure 1. Further details of the processes involved are described in the following sections.

Figure 1: Process and factors determining the rating of heating sources

Thumbnail of image. View figure at its full size (including text description).

Overview of the operations in the Excel model

This section presents an overview of the operation of the Excel heater rating model, and the following sections describe in more detail these operations and why they are undertaken. The model consists of a number of worksheets containing information on the following characteristics.

  • Heating scenario defined: the user specifies the characteristics of the dwelling to be heated and the relevant heating behaviours they are interested in, using options selected from pull-down menus in the Scenario worksheet.
  • Weighting of heating attributes: the user specifies the importance they place on different aspects of the performance of the heaters by assigning a percentage weighting to different criteria using the Weighting worksheet. Default values are assigned if the user does not elect to assign weightings.
  • Heat energy requirement determined: to determine the operating costs, energy consumption and emissions of different heaters in the different heating scenarios, it is necessary to determine the annual heating energy required in the different scenarios. This is done by using values derived from the heating scenario to look up the annual heat energy requirement in the Heater Energy Table. The results from the Heater Energy Table are placed in the Heating Energy Requirement worksheet and a series of revision factors are applied to the result, if required, to allow for variations in house size, use of insulation and other factors. These revisions produce the final annual heat energy requirement.
  • Heater output requirement determined: the suitability of the different heaters to satisfy a defined heating scenario will, to a large extent, depend on whether the heater's output is sufficient to meet the peak heating demand of the room or house being heated, and also whether the heater is oversized for the heating requirement. It is therefore necessary to determine the peak heating output required in the different scenarios. This is done by using values derived from the heating scenario to look up the peak heat output requirement in the Heater Output Table. The results from the Heater Output Table are placed in the Heating Output Requirement worksheet, and a series of revision factors are applied to the result, if required, to allow for variations in house size, use of insulation and other factors. These revisions produce the final heat output requirement.
  • Performance of heaters: in the Appliances worksheet the performance measures of all the modelled heating options are calculated using the known characteristics of the heaters, their fuels and the annual heat energy requirement. Their suitability for the heating scenario is also determined by rating their suitability to heat a single room or the whole house being modelled. If they receive a low rating, then the heater will be treated as unsuitable for the purpose required.
  • Heater information: the key attributes and performance measures are extracted from the Appliances worksheet and displayed in the Heater Information worksheet, where they can be viewed by the model user.
  • Heater ratings: the ratings of the heaters on nine attributes are derived from the attributes and performance measures prepared in the Appliances worksheet. Better performances are assigned a higher rating, with the ratings ranging from 1 to 5. These rating values are displayed in the Heater Ratings worksheet. An overall weighted rating is assigned to each heater, which is calculated from the ratings on the nine attributes and the percentage weighting given to each attribute by the user in the Weighting worksheet. The heater is also assessed as being suitable or not suitable to fulfil the user's heating scenario based on the known attributes of the heaters, such as their heating capacity. Unsuitable heaters are given an overall rating of zero.
  • Rating results: the final process undertaken by the model is to copy the heater ratings from the Heater Rating worksheet and sort them by overall ratings, with the highest-rated heaters displayed at the top of the table.

Heating energy requirements

Dwelling characteristics

The heating requirements of the householder will be influenced by the physical characteristics of the dwelling or living space that is to be heated. The physical characteristics considered most relevant include the:

  • climate in which the dwelling is located
  • type of dwelling
  • insulation of the dwelling
  • size of the dwelling.

The evaluation model needs to allow the user to select the characteristics of the dwelling they are interested in and then produce an estimate of the heating energy requirement for that type of dwelling. The model does this by enabling the user to select their dwelling characteristics and then, by looking up a table contained in the model, the heat energy required for the selected type of dwelling. The information in this table is discussed in Information on annual heating energy requirements.

The evaluation model allows the user to select the characteristics of the dwelling they are interested in by choosing from a series of options for different types of characteristics. In the Excel model these options are chosen by the user in the Scenario worksheet. This sheet presents the user with a heating scenario table. The table headings include:

  • Geographic Area
  • City or Regional/Rural
  • Natural Gas Available
  • Type of House
  • Insulation
  • Heating Behaviour
  • Size of House
  • Whole of House vs Main Living Room.

Under the table headings the user can choose one of several pull-down menus. For example, under Type of House, whether their dwelling is a single-storeyed house, a two-storeyed house, or a flat/apartment; and under Geographic Area the user can choose if their dwelling is located in the Northern, Central or Southern parts of New Zealand. There is also a Map Regions worksheet, which displays the three geographic areas as they relate to heating requirements.

Once the Excel model user has selected their heating scenario, they can press the 'Default Ratings Display' button. This will operate a macro programme that takes the user to a Rating Results worksheet, which displays the heating options sorted by their overall suitability rating for the heating scenario chosen. Next to the Rating Results worksheet is the Heater Information worksheet, which displays more detailed information on the performance of the heating options under that heating scenario.

Other factors could be added to this list of physical characteristics of the dwelling, such as the amount of glazing in the dwelling, the mass of the dwelling (for heat retention), and the orientation of the dwelling. However, the present factors were considered the most relevant for the present evaluation model, and the amount of glazing and mass of the house were assumed to be medium in the Excel model. These assumptions are listed at the bottom of the Scenario worksheet. It was also assumed that when heating a single room in a dwelling, the heating energy required would be 40% of the total energy required to heat the dwelling, which is approximately what would be required if the single room was around 25% of the area of the dwelling.

Four of the scenario headings do not refer to the dwelling's physical characteristics but are related to householder heating behaviour, fuel availability or cost. These include the headings 'Natural Gas Available', 'City or Regional/Rural', 'Heating Behaviour' and 'Whole of House vs Main Living Room'. The use of these variables will be discussed below.

Householder behaviour

The physical characteristics of the dwelling to be heated obviously have a major influence on the final amount and type of heating required by a household, but the householder's heating behaviour also significantly affects their heating needs. Some important heating behaviour variables that affect heating requirements are:

  • the decision to heat the entire dwelling or just a main living space
  • the length of time heating is required, and during what part of the day or night.

The householder's decision on both of these factors will affect the final heating requirement and may influence the ability of different heating sources to deliver the heating required. For example, a single heat pump may be more than adequate to heat a single room but may be inadequate for heating an entire house, depending on its size and layout.

As previously discussed, in the Excel model these options are chosen by the user in the Scenario worksheet. The relevant headings in the heating scenario table are:

  • Heating Behaviour
  • Whole of House vs Main Living Room.

The Heating Behaviour pull-down menu offers the user the following choices concerning the amount of time they want their house heated:

  • 24 hours a day
  • day and evening
  • morning and evening
  • evening only.

The other pull-down menu simply offers 'Whole House' and 'Main Living Room'.

Other heating behaviours, such as the extent the householder is concerned about convenience versus costs, are also incorporated into the model, but this is done via the weighting different criteria are given when rating the heaters. This is discussed in Heater overall weighted rating and suitability.

Information on annual heating energy requirements

As previously mentioned, the Excel model uses information in a table on the annual heating energy requirements of different types of dwellings. The information in the table comes from estimates of the annual heat energy demand of 'typical' houses with a variety of characteristics. There are numerous ways of preparing these estimates, but generally they are done with computer models of these so-called typical houses.

The estimation of such heating demand has been conducted by the Energy Efficiency and Conservation Authority (EECA), and this information source was used for the fundamental data on physical heating requirements for developing the Excel heating rating model. [M Donn, G Thomas, Designing Comfortable Homes, Energy Efficiency and Conservation Authority and the Cement and Concrete Association of New Zealand, Wellington, 2001.] EECA provides information on the annual heating requirement for typical modern two-storeyed and single-storeyed houses in the climates of Auckland, Wellington and Christchurch. The heating requirements of the houses modelled were determined on the assumption that the living areas of the dwellings would be heated to at least 20°C and the bedrooms to at least 16ºC. The houses are varied on a number of attributes to produce a range of modelled houses and estimates of their resulting annual heating energy requirements. The attributes used include:

  • extent of insulation
  • construction mass
  • extent of glazing
  • timing of heating availability: 24 hours a day or 7 am to 11 pm.

The extent of insulation was divided into three types:

  • Code Compliance, which assumes a slab floor (R1.7), walls R1.8 and roof R1.8 or R2.1 in the Southern zone
  • Good Practice, which assumes a slab floor and edge insulation (R1.9), walls R2.0, roof R2.4 and double-glazed windows
  • Best Practice, which assumes a slab floor and under-slab and edge insulation (R3.1), walls R2.8, roof R3.2 and double glazed windows.

To use this information in the Excel heating rating model, a number of assumptions needed to be made concerning the characteristics of the dwellings whose heating requirements were to be determined by the model. The key assumptions were that:

  • construction mass would be assumed to be low
  • the extent of glazing would be treated as medium
  • the three cities modelled would have similar climates to their surrounding three regions, hence the use of the Northern, Central and Southern zones as defined by the New Zealand Standard for energy efficiency of houses (NZS 4218:1996).

In addition, the size of house modelled to produce the heating requirement data was 200 m2 and this was assumed to represent the mid-range of a medium-sized house. Small houses were assumed to be less than 150 m2 in size and larger houses were assumed to be greater than 250 m2.

Having made these assumptions, a table of annual heat energy requirements for a range of houses could be developed from the EECA's information and input to the Excel model. This table is contained in the worksheet Heating Energy Table. Likewise, a table of the peak load heating requirement for the range of houses could be developed from the EECA's information and input to the Excel model in the worksheet Heater Output Table. This table lists the estimated heat output required to heat the house to the required comfort levels during the coldest periods experienced in the relevant geographic region in which the dwelling is located.

The EECA information on annual heating requirements, and the resulting Heat Table, formed the basis for estimating the required heating energy for different households in the Excel model. However, these estimates needed to be modified when the housing scenario being modelled did not fit the typical houses documented by EECA. This could occur when the Excel model user chose a heating scenario that had any of the following characteristics:

  • insulation was not present, or was below code standards
  • the house size was smaller or greater than the medium-sized house used by EECA
  • householders were planning to heat only one living space rather than the whole house
  • the time the household wished to heat their dwelling was less than 18 hours daily.

In each of these situations the heating requirement determined using EECA's information needed to be revised. To do this a series of revision factors was included in the Excel model, which increased or decreased the estimated heating energy requirement. These revision factors are presented in the worksheet Heating Requirement, and lead to a scaling up or down of the relevant heating requirement obtained from the Heat Table. The revision factors were mainly deduced from information gathered from Environment Canterbury's Clean Heat Project concerning heating requirements with and without insulation in different-sized living spaces, and from the experience of EnergyConsult and Strategic Energy.

Rating of heating options

Attributes of heaters

For any given household, the ability of each type of heater to supply the required heat may vary due to factors such as the heating capacity, fuel requirements and design of the heater. Consequently, the attributes of the heaters need to be known and incorporated into the Excel heating rating model so the model can determine the performance of the different heaters in meeting the heating requirement.

Phase 1 of the Warm Homes options project involved a review of the current information on home heating options, and has resulted in a report of the various attributes and features of the different types of heaters. The information in that report forms the basis for the analysis of heating options performance in the Excel model. The information was incorporated in the model by developing a worksheet, called Appliances, which contains a table with fixed attributes as well as derived performance outcomes for each heating option.

The fixed attributes for each heater are:

  • fuel type
  • heating capacity
  • thermal efficiency
  • life of the appliance (average in years)
  • capital costs
  • PM10 emissions per unit of fuel consumed
  • convenience
  • ability to heat a whole house
  • suitability to heat a main living room
  • comments on ability to meet heating requirements.

It should be noted that the lower end of the price range for capital costs was used to determine the capital cost figure used in the Excel model. This was done because generally the higher costs of heaters of the same heat output reflect greater functionality or market branding, attributes that were not central to what the Excel model was trying to assess. Also, many heater prices appear to be dropping, so using the lower end of the price ranges may help to keep the model relevant to the market for a longer period.

The following performance outcomes were calculated from the fixed attributes of the heaters, from the previously determined annual heat energy requirement and from the attributes of the fuel/energy used by each heating option:

  • energy required per year (in kilowatt hours – kWh)
  • operating cost per year
  • average lifetime cost per year
  • PM10 (particulate) emissions (in kilograms per year)
  • greenhouse emissions (in kilograms per year).

Both the heating option attributes and the calculated performance attributes are displayed in the Appliances worksheet.

Fuel availability, costs and attributes

One of the key determinants of whether a heating option would be suitable for a particular dwelling will be the availability of the fuel it uses. Of the fuels used by the heating options modelled, there was only one fuel, reticulated natural gas, which had limited availability. In the model the user simply indicates whether there is natural gas available to the dwelling by choosing 'Yes' or 'No' from the Natural Gas Available menu in the Scenario worksheet. This information is then incorporated in the Excel model using a series of formulae which tag heaters using natural gas as 'Unsuitable' in the heating option rating process if natural gas is not available.

The information on the costs of the different fuels was principally collected via the literature review conducted in Phase 1 of the Warm Homes options project. Costs were collected for fuels in whatever unit the fuel was sold in but then converted into a cost per kWh of energy delivered. It should be noted that fuel costs are average fuel costs, estimated using information on minimum and maximum costs, wherever possible. The fuel cost information is contained in the Fuels and Wood Costs worksheets. The costs are then used by the Appliances worksheet in calculating operating costs.

The Excel model also recognises that city and regional/rural electricity costs may vary. The model asks in the Scenario worksheet whether the dwelling is in the city or a regional/rural area. Depending on the option selected by the model user, either city or regional/rural electricity costs will be used to derive operating costs for electrical heaters in the Appliances worksheet. We recognise that there is wide variation in electricity prices across New Zealand and that tariffs can be quite complex. The Excel model does not try to incorporate this complexity but rather uses approximations for the medium prices in both city and regional/rural areas.

The PM10 emission characteristics of fuels were also researched in Phase 1 of the Warm Homes options project and are included in the Fuels worksheet. These values are used by the Appliances worksheet to calculate kg per year of PM10 emissions from the different heaters under the different heating scenarios. These results are calculated by deriving the energy required by each type of heater under each scenario, then calculating the fuel they would use and from this determining the PM10 emissions that will be produced by burning that amount of fuel.

The greenhouse emissions for each heating option under the various heating scenarios are also calculated by the model. These emissions are determined by deriving the energy required by each type of heater under each scenario, then multiplying the energy used by the relevant greenhouse emission factor for that energy/fuel type. The greenhouse emission factors were all sourced from the Ministry for Environment website, with the exception of the factor for wood and pellet burning, which was sourced from the Australian Greenhouse Office's Methods and FactorsWorkbook. [Australian Greenhouse Office,Methods and Factors Workbook, Canberra, 2004.]

Performance and ratings of heating options

As described above, the fixed attributes of the heaters and the performance attributes that vary with the household's heating requirement are presented in the Appliances worksheet. The Excel model then produces two forms of information on the heaters: information about the heaters' performance under the heating scenario, and ratings of the heaters.

Information about the heaters and their performance is created in a table in Heater Information, and the underlying information used to create this table all comes from the Appliances worksheet. The Heater Information table contains detailed and mainly quantitative information on the performance of the different heating options under the particular heating scenario that has been run. It also contains comments on the suitability of the heaters to heat the whole house, more than one room, etc.

The ratings of the heating options are created from information in the Appliances worksheet and are presented in a table in Heater Ratings. The ratings are created by examining the relevant attribute of each heater and assigning a rating from 1 to 5, depending on the value of the attribute. A 5 rating indicates the heater has an excellent performance on the attribute being considered, while a 1 rating indicates the heater has a poor performance on that attribute. The rating thresholds are shown in Table 1.

The Excel model uses these ratings through the use of a look-up function. This function compares the heater's characteristics (eg, operating cost) with values in the left column of the table and determines the highest value in this column where the heater's value is less than the value in the left column. The function then assigns the rating value from the right column to the relevant heater attribute. So, for example, if the heater has an annual operating cost of less than $500 the function would assign a 5 rating to the heater, but if the heater operates at between $501 and $750 per year then it would be assigned a 4.5 rating. (Note: the tables in the rating model are set up slightly differently due to the way the look-up function works).

The exception to this is the capital costs rating, which the Excel model calculates in a slightly different way. If the capital costs of the heater are less than $500 it is rated a 5, and if they are over $15,000 then it is rated a 1. In between these capital costs the rating value assigned is determined according to a linear function whereby for each $3,625 addition in the capital cost above $500, the rating assigned decreases from 5 a further one rating point.

Table 1: Threshold values for determining cost ratings

View threshold values for determining cost ratings (large table)

The Heater Information table displays information, and the Heating Rating table produces a rating for each of the following attributes of all the heating options:

  • operating costs
  • capital costs
  • average lifetime costs
  • thermal efficiency
  • energy usage per year
  • greenhouse emissions
  • particulate (PM10) emissions
  • convenience
  • suitability to heat a single room
  • ability to heat the whole house
  • comments on the ability to meet heating requirements.

In addition, the Heater Information table indicates the heating capacity of each heating option.

Heater overall weighted rating and suitability

It was decided that the Excel model would produce an overall rating of the different heating options to guide the user in selecting a heater. The model also needed to indicate to the user whether each heater type was suitable for the heating scenario developed by the user.

To produce an overall rating for the heating options it is necessary to know the user's heating priorities; in other words, the emphasis the user places on different aspects of the heater's performance and attributes. For example, if the only things important to the user are operating costs and convenience, then only these two attributes should be considered in developing an overall rating for the heaters. However, if the user's concern is with the particulate and greenhouse emissions of the heaters, then it would be these two attributes that would be considered in determining the overall rating.

The Excel model can obtain information on the heating priorities of the model's user if the user elects to complete the Rating Weighting Table in the Weighting worksheet. The user can assign a weighting value from 0% to 100% to any of the listed rating criteria, and the total value of the assigned weightings must equal 100%. The rating criteria include:

  • operation cost
  • capital cost
  • lifetime cost
  • thermal efficiency
  • energy use
  • greenhouse emissions
  • outdoor air quality impacts (PM10)
  • convenience
  • ability to heat the whole house.

If the user does elect to assign their own rating weightings then they must complete the heating scenario they wish to test in the Scenario worksheet, and then go to the Weighting worksheet and assign their weightings. Having assigned their weightings, they then click on the 'Weighted Ratings Display' macro button and the Rating Results worksheet will appear.

Alternatively, if the user elects not to assign their own weightings to the different ratings criteria, then a set of default ratings weighting will be used by the model. This is performed when the user clicks the 'Default Ratings Display' macro button on the Scenario worksheet.

The overall rating is produced for each heating option in the Heater Rating worksheet by multiplying each rating result displayed in the Heating Rating worksheet by the relevant percentage weighting that has been assigned to that rating in the Weighting worksheet. For example, an operating cost rating of 3 in the Heater Rating worksheet might be multiplied by 50% from the Rating Weighting Table, and a capital cost rating of 1 might also be multiplied by 50% to produce an overall weighted rating of 2. The overall rating would then be displayed in the relevant column of the table in the Heater Rating worksheet.

The overall suitability of the heating options is also assessed in the Heating Rating worksheet. The heating options are all assumed to be suitable unless one or more of the following occurs, in which case the heater is rated as not suitable.

The heater uses natural gas and this is not available according to the heating scenario.

The heater has a 'Low' assessment in terms of its suitability to heat the whole house, as shown in the Appliances worksheet, and the user wishes to heat the whole house. This will occur if the heater can not produce within 15% of the peak heating requirement the model indicates is required for the relevant heating scenario.

The heater has a 'Low' assessment in terms of its ability to heat a single room, as shown in the Appliances worksheet, and the user wishes to heat only a single room. This will occur if the heater can not produce within 15% of the peak heating requirement the model indicates is required for the relevant heating scenario, or if the heater produces over double the peak heating requirement the model indicates is required to heat the room.

If the heater is assigned a 'Not suitable' rating, then this is displayed in the Heating Rating worksheet. In addition, the overall rating for the heater is assigned a zero rating.

Displaying the model results

All the ratings and information on the heating options have been prepared by the Excel heater rating model at this stage, but the model undertakes one more procedure to better present the analysis results. A programme macro copies the values of the heater ratings in the table in the Heating Rating worksheet into a new worksheet, the Rating Results worksheet. This new table is then sorted by the overall rating of each of the heater options and is displayed with the highest ratings at the top of the table. This is the table the user can see when they click the macro button in either the Scenario or Weighting worksheets.

The other form in which the model results are displayed is the heater performance information in the Heater Information worksheet. The user can access this information by moving to this worksheet from the Rating Results worksheet.

Using the heater rating model and literature review

The heater rating model and the report on the literature review of the heating options form the main outputs of the Warm Homes options project. These are the tools the project has developed to assist the Ministry for the Environment to develop policy and advise the public.

The Excel heater rating model is a tool that will enable Ministry for the Environment staff to explore the varying heating options that are best suited for different types of dwellings across the country. The model enables over 1700 heating scenarios to be examined and also allows for these to be explored using different priorities or weightings regarding the performance of the heaters. For example, the model might be run with a high performance weighting given to heaters with low particulate emissions, to heaters with low operating costs, or to heaters with low greenhouse gas emissions.

The model is well suited to helping Ministry for the Environment staff advise the public on the heating options that might best suit them, and on the potential costs and other implications of their heating choices. It may also be useful for assisting other government organisations to select heating for public housing or for community housing improvement programmes. The model's information can be supplemented by the more detailed information available from the literature review of the heating options.

The model can also be used to explore policy options and implications. Because the model will produce a different list of heating options, sorted by their overall rating, depending on the performance weighting used, the different results from different policy priorities can be compared. For example, a policy to reduce the use of high particulate emission heaters can be explored by operating the model with a high weighting given to low PM10 emission heaters. The recommended heaters can be noted, together with the information on their various performance attributes, then the model run again with the model weighted for a different performance weighting, such as low operating costs. The difference in the heaters that are given a high overall rating can be obtained, and the difference in their performance on relevant attributes compared. This will enable at least some of the impacts of different policy options on different type of dwellings (and hence communities) to be compared. Again, these outcomes from the model can be supplemented with information from the literature review.

By operating the model with a variety of heating scenarios and with varying weightings on performance criteria, and by using the information from the literature review, the Ministry for the Environment will be able to gain greater insight into the heating options relevant to different housing types.

[ | ]