When we analysed the Minister’s speech on the new seismic strengthening policies (‘The Minister’s new clothes’), we commented on the high estimate of lives saved over the next 100 years. The Minister’s estimate of 330 did not make sense compared to the estimate of 24 used in MBIE’s cost benefit analysis.
To get to the bottom of the difference, we asked MBIE on 21 May 2015, under the OIA, for all documents that would explain the new estimates. We were initially told that to provide the relevant documents would be a “contempt of the House of Representatives” (‘The dog ate my homework’). Then MBIE needed more time. Finally last Thursday, (nearly three months after the request was made) the documents arrived.
But there was no document that explained the calculation. Instead were we referred to the Cabinet paper where the following statement was made.
“Based on population adjusted historical fatalities in New Zealand across New Zealand it is estimated that the new policies will result in 335 fewer fatalities and 368 fewer serious injuries over the next 100 years.”
Apparently, if the OIA response is to be believed, there is no working document where these calculations were made, so it appears someone did the mental arithmetic, which then went directly into the Cabinet paper.
The estimate would have looked to be scientific and authoritative to Cabinet, who no doubt would have would have looked askance at a policy that was causing so much cost and trouble, if the true figure of just 24 fewer fatalities had been put to them.
Without a document trail we cannot be sure where the 330 or 335 fewer fatalities came from, but we have a couple of theories. The first is that it was taken from the Martin Jenkins cost benefit analysis report. An analysis was done that assumed that all New Zealand cities have the same seismicity as Wellington. The results, reported on page 37, showed average deaths, of 8.07 per year without strengthening and 5.76 with strengthening to 34% NBS. On page 43 of the report it is stated that:
“A modified sensitivity produces 8.5 deaths p.a. and total benefits on this basis are shown in the last block on the chart. This sensitivity (for benefits) is more in line with actual historical deaths, adjusted for population growth.”
The language here resembles that in the Cabinet paper so this may be the source. If we take the difference between 8.5 and 5.76 (2.74) and multiply by 100 that gives us 274 lives saves. Close, but still 55 lives unexplained.
The second theory is that they took the number of deaths in earthquakes around 1930 (a total of 274 for Murchison, Hawkes Bay and Pahiatua), multiplied by three to adjust for New Zealand’s population growth, then added Christchurch (185) to get a lives ‘lost’ total of 1007. If we multiply this figure by .33 (because strengthening to 34% NBS will on average reduce deaths in earthquakes by one third) we get a lives saved result of 332, which is right between the Cabinet and speech figures.
Either way, the 330-335 lives saved estimate was a misleading fabrication. On the first theory, the assumption that the whole of the New Zealand has the same seismic risk as Wellington is plainly wrong. On the second, there are obvious issues. The number is dominated by the inflation of the Hawkes Bay earthquake death numbers, but this takes no account of the fundamental change in the New Zealand building stock since then; the movement in population to very low risk seismic zones, and the fact that the two major earthquake disasters over the past hundred years does not reflect the average odds.
In the Minister’s speech it was also said that 80 percent of lives saved (264) saved would in Wellington. The true number was 19, but Wellingtonians were given a grossly misleading impression of their risk when the speech was reported in the Dominion Post.
The obvious fabrication of evidence to promote the new seismic strengthening policies should be an embarrassment to those responsible for it, and it comes as no surprise that MBIE have gone to some lengths to cover it up.
The fundamental justification for the policy is saving lives. Here we are told that the policy will result in an estimated 330 fewer deaths over the next century. When Martin Jenkins and GNS Science did the analysis for the Ministry in 2012 they found that the average lives lost to earthquakes in New Zealand per year was 0.96. Reaching the 34%NBS standard reduced this number to 0.72.
On our arithmetic this equates to 24 lives saved over the next century (the actual number will be less because even without strengthening requirements a large number existing of buildings will naturally be replaced with stronger modern ones).
Quite how the Minister came up with the figure of 330 fatalities (of which 264 will be in Wellington) is a mystery at this point.
We will be asking the Minister and MBIE under the Official Information Act to provide the documents that explain where the 330 fatality number comes from. Our analysis of the documents will be shared on this site.