The ever receding yet

Historian Niall Ferguson has been reported as apologising for remarks in which he implied that John Maynard Keynes did not care about future generations because he was childless and gay leaving open the cause of Keynes indifference to the long run. This provoked a letter from me to todays Guardian. For non-Guardian readers here it is.

Niall Ferguson apologises

 How embarrassing. Forget the homophobia. The renowned historian appears to be unaware of Keynes’s famous grandchildren (Report, 4 May). In an essay written in 1930, entitled Economic Possibilities for our Grandchildren, Keynes marvels at the power of compound interest, and invites the reader to join him in pondering the impossibility of it compounding forever. He imagines the advent of “an age of leisure and abundance” in which his (hypothetical) grandchildren now well into retirement will be set free “to return to some of the most sure and certain principles of religion and traditional virtue, that avarice is a vice, that the exaction of usury is a misdemeanour, and the love of money is detestable”. But, writing at the beginning of the Great Depression, Keynes warns: “Beware! The time for this is not yet.”

What unites today’s economists, Keynesian or not, is the conviction that we are still a long way from “yet”. The central challenge being addressed by governments everywhere is how to get economic growth restarted, and then growing faster without apparent end. If not yet, when?

The Boston Marathon Bombs

In the aftermath of the 2005 7th July bombings in London I wrote a piece entitled “7/7: What kills you matters – not numbers

I illustrated it with a diagram highlighting the remarkable lack of correlation between quantified measures of risk and common response. I identified two key variables that helped to explain this lack of correlation: the level of control that those taking, or exposed to, the risk felt that they had and, in the case of involuntary risks, the perceived motives of the imposer of the risk.

Risk Amplification

Screen Shot 2013-04-17 at 21.52.23

52 people were killed by the 7/7 bombers. I noted that in Britain at that time on an average day nine people die and over 800 are injured in road accidents. The mangled metal, the pain of the victims, and the grief of families and friends, one might suppose, are similar in both cases. Measured in terms of life and limb, 7/7 represented six days of death on the road. But thousands do not gather weekly in Trafalgar Square to manifest their collective concern.  Why?

Simon Jenkins has become a routinely calming and rational voice in the aftermath of such events. Discussing the Boston marathon bombs he provides a good answer in todays Guardian. He succinctly describes the problem: Such deeds are senseless murders. What makes them terrorist is the outside world rushing to hand their perpetrators a megaphone. See After the bomb, hysteria is the terrorists best weapon. I recommend it.

Another recommendation from yesterdays New York Times by Thomas Friedman: Bring On the Next Marathon.

Now wash your hands

I recently visited an exhibition at the Barbican Centre in London entitled The Bride and the Bachelors: Duchamp with Cage, Cunningham, Rauschenberg and Johns

More recently I was invited to speak to a conference entitled Risk culture for charities organised by the Institute for Risk Management.

I began my conference presentation with an overview of disparate risk-management cultures, offering examples of excessive risk aversion in the form of fatuous warnings of manifestly obvious directly perceptible risk. And I concluded with a cartoon that I felt made the point rather well.

brick wall

                                                                                         Guindon, Detroit Free Press, 25 September 1994

As the profusion of signs warning of obvious and/or trivial risks grows, I suggested, the likelihood increases that warnings of serious/non-obvious risks will be ignored.

After my presentation, and after tea, one of the conference participants returned to report that every urinal in the mens toilets displayed the injunction Now wash your hands.*

The centrepiece of the Barbicans Duchamp exhibition was his famous urinal – signed R. Mutt 1917.  I have long had trouble with Duchamps success in transforming it into a work of art.  But work of art, or no, it clearly needs updating to keep up with modern risk management practice.

duchamp hse

* He also reported that the man next to him departed without washing his hands.

 

 

 

The BIGGEST LIE?

Last Thursday (21 March 2013) I attended a conference entitled Lies, Damned Lies and Statistics? Understanding casualty trends and the causes. It was sponsored by the Parliamentary Advisory Council for Transport Safety (PACTS). At the conference PACTS, the Department for Transport, and seven other organizations interested in promoting road safety launched a website called www.roadsafetyobservatory.com.

Its home page promises:

Key facts and summaries of research on road safety topics

The website proclaims:

“The Road Safety Observatory aims to provide easy access to independent road safety research and information for anyone working in road safety and for members of the public. It provides summaries and reviews of research on a wide range of road safety issues, along with links to original road safety research reports.

Its objective, the person presenting it made clear, was to provide an authoritatively fact-checked set of statistics to inform public discussion of road safety issues. Under the heading How is the Research Evidence Produced? it states:

The Programme Board commissions Research Reviews from a wide range of researchers and road safety experts. Each Review is then submitted to an independent Evidence Review Panel, who must approve it for publication on the Observatory website. The reviews are intended to be free from bias and independent of Government policies and the policies of the individual organisations on the Programme Board. They represent a summary of the best evidence readily available to the research community and will be kept under review as new evidence emerges.

I decided to subject this new source of evidence to my standard test. Into the Home Page search bar I typed seat belts do try it at home. This took me to “Road Safety Observatory Seat Belts Review, January 2013”  bang up to date and approved by the Independent Evidence Review Panel. First up came “SEAT BELTS: KEY FACTS”.

Seven bullet points down I found this Key Fact: By January 2008, 25 years after the introduction of the law, front seat belts had saved over 60,000 lives in Great Britain.

On page 5 the claim is embellished: By January 2008, 25 years after the introduction of the law requiring car drivers and front seat passengers to wear seat belts (if fitted), it was estimated that front seat belts had saved over 60,000 lives in Great Britain.

On page 9 the claim is repeated twice.

And again on page 32: Seat belt laws increase seat belt use, and so reduce death and injury. 25 years after the first law requiring seat be to be used, it was estimated that front seat belts had saved over 60,000 lives in Great Britain.

The claim that the seat belt law saved 60,000 lives over 25 years is reiterated five times in a document approved by the Observatorys Independent Evidence Review Panel without any supporting citation I could find.

Citations are no longer needed. The claim has been repeated so often that it has become a fact that everyone knows. Typing seat belts 60,000 lives saved into Google yielded, at last clicking, over one million hits.

THE CLAIM IS OUTRAGEOUS NONSENSE.  According to this claim the seat belt law is, by far, the most effective road safety measure ever implemented in Britain. 60,000 divided by 25 years would equal 2400 lives saved every year since 1983 by the seat belt law.  In 1982, the year before the seat belt law came into effect there were only 2365 driver and front seat passenger fatalities! The claim is further, if less dramatically, contradicted by other claims in the same review indeed in the same initial bullet point: It was estimated that the seat belt law saved the lives of 241 drivers and 147 front passengers in 1983 and 270 drivers and 181 front passengers in 1984.   The review does not provide a citation for the estimate of lives saved in 1983 and 1984 but they are similar to those provided in a June 2008 article in Significance by Richard Allsop, Oliver Carsten, Andrew Evans, and Robert Gifford (all members of PACTS). See table below.

Significantly the Significance article did not make it into the Reviews list of key references on seat belts. A significant omission because the authors, all defenders of the seat belt law, acknowledge an effect of the law of important consequence to vulnerable road users. They say the clear reduction in death and injury to car occupants is appreciably offset by extra deaths among pedestrians and cyclists.

seatbelts2

This surely deserves a place in any list of Key Facts relating to the seat belt law.

Another Key Fact that did not make it into the review is the finding that almost all of the 1983 decrease in driver fatalities estimated by the authors of the Significance article took place between 10pm and 4am (the drink drive hours) and in drivers with alcohol in their blood. During the other hours of the day, and amongst alcohol-free drivers there was no detectable departure from trend (see Risk chapter 7). 1983 also happened to be the year in which evidential breath testing was introduced, and unprecedented numbers of breath tests were administered, and drink-drivers prosecuted. The modest claims (modest when compared to 2400) for lives saved in the table above can only be attributed to the seat belt law if one assumes that the measures introduced by the campaign against drink driving launched in the same year had no effect.

Who cares? Why does all this matter 30 years after the seat belt law came into effect? It matters because a new website sponsored by what might be called Britains road safety establishment, and promising unvarnished, unbiased, objective Key Facts, is telling a whopper. And not out of ignorance. The last time I looked PACTS, RoSPA and the DfT, key sponsors of the new website, all had the 60,000 claim still on their websites despite its absurdity having been called to their attention on a number of occasions.

The PACTS conference was well attended by people with a common interest in making roads safer. There was a detectable, and welcome, shift in emphasis from previous conferences I have attended from a focus on legal and engineering approaches to the problem, to an interest in changing attitudes and behaviour. As I have noted in   Managing transport risks: what works?  :

 There are two different kinds of manager involved in the management of transport risks: there are the official, institutional, risk managers who strive incessantly to make the systems for which they are responsible safer, and there are the billions of individual fallible human users of the systems, each balancing the rewards of risk against the potential accident risks associated with their behaviour.

 Understanding the attitudes of the latter to risk, and influencing their consequent behavior, represents the most important and challenging part of the institutional risk managers job.

 Seat belts have acquired iconic status in common parlance. Fasten your seat belts is a phrase commonly invoked to introduce any exciting or threatening idea into a discussion. It is routinely used as a comparator for the postulated benefits of other forms of safety legislation such as cycle helmets. It reinforces legislator hubris look what we have achieved! they marvel as they pat themselves on the back. The Road Safety Observatory by virtue of its endorsement of the 60,000-lives-saved claim is further entrenching the iconic status of the seat belt law and energizing legislators keen to pass further well-meaning but misguided legislation.

 To undo this damage PACTS, RoSPA and the DfT need not just quietly remove the claim from their websites, but prominently explain why.  Likely to happen? Dont think so.

 Recent posts and comments on this subject are on my website at http://www.john-adams.co.uk/category/seat-belts/

 

Repeal the Seat Belt Law

Peter Bonisch  has posted a comment on my RoSPA post (11 February) that merits an answer. He asks: would you advocate now removing the seatbelt requirement?  The world has changed since the law was introduced including having been changed by it.  Would it now be constructive to abolish the law in the knowledge that we would not simply revert to the ex ante position?

Good questions.

I would not expect to return to the ex ante position. And yes, I would repeal the law.

Over the last 30 years road accident deaths in Great Britain have fallen from 5934 to under 2000, but the clear downward trend is not a demonstrable consequence of any government safety interventions over that time see Managing transport risks: what works, Figures 1, 2 and 3.

The long-term trend in the ratio of pedestrian and cyclist fatalities to car occupant fatalities has also displayed a downward trend as car traffic has increased and the number of vulnerable road users (cyclists and pedestrians) on the road has decreased with a significant interruption to the trend in the year that that the seat belt law was implemented – see p. 10 of what works. An important contributor to this trend has been the enormous drop in the number of children allowed on the streets on foot or on bicycle without adult supervision see Children’s independent mobility: a comparative study in England and Germany (1971-2010) 

Bonisch notes that there have been technological advances in safety over the past 30 years, such as airbags, improved braking and anti-collision technology. But these changes appear to account for very little, if any, of the decline in road death rates with increasing levels of motorisation. Today motorists in poor countries in the early stages of motorisation are achieving kill rates per vehicle driving  vehicles with one hundred years of safety technology built into them – as high or higher than were achieved by Model-Ts in the early 19th century. The probable reasons for this are explored in Where and when is shared space safe? 

Bonisch contends that the world has been changed by the seat belt law, and I agree. It has shifted the burden of risk on the road unfairly from the best protected in cars to the most vulnerable outside cars.

Finally, I rest my case on John Stuart Mills famous dictum opposing measures that would compel people to be safer than they voluntarily choose to be: “all errors which he is likely to commit against advice and warning, are far outweighed by the evil of allowing others to constrain him to what they deem his good”.

I would repeal the law.

 

Open letter to Tom Mullarkey, CEO of the Royal Society for the Prevention of Accidents.

Dear Tom

Following our meeting at a dinner at the RSA over six years ago I sent you an email (22 February 2007).You replied the same day saying When I have a moment, I would like to look into this in more detail and so I will follow up on the links you have kindly provided.

I guess youve been busy.

In any event RoSPAs press release on 31 January 2013 celebrating the 30th anniversary of the seat belt law, and RoSPAs contribution to it, have given me an excuse to revisit the issue.

At dinner I had told you about RoSPAs dubious lobbying practices in the run-up to the Parliamentary debate in 1981 and promised to send you some chapter and verse. In my 22 February email I noted that I had already detailed some of this dubious practice in my book Risk and Freedom (p152). I quoted from my book:

“On July 7 1981 The Royal Society for the Prevention of Accidents (president – Lord Nugent) sent a letter to every member of Parliament stating

‘Dr. Adams has recently published a paper advancing the thesis that the wearing of seatbelts may actually increase road accidents by encouraging a false sense of security. His paper [subsequently peer-reviewed and published unaltered by the American Society of Automotive Engineers]  presents road accident trends in several foreign countries to support his view. His paper requires detailed analysis with the aid of much background information from the countries concerned before an authoritative comment can be made upon it. RoSPA and the Transport and Road Research Laboratory are undertaking these studies, meantime it is relevant to record that Dr. Adams does unequivocally state that wearing seatbelts greatly improves the chance of avoiding injury.’ “

This last sentence, I noted, unequivocally misrepresented my position. As I said at dinner, the statistics demonstrate that a seat belt improves your chance of surviving a crash, but INCREASES your chances of being in one. I went on to say

“This letter throws a revealing light on the campaigning methods of RoSPA at the time. While publicly stating that much more work needed doing before they could comment authoritatively on my findings, privately they had been briefing selected members of Parliament for three months with a document which asserted confidently that my conclusions were ‘absurd’ and ‘without foundation’ and that I ‘exhibited, at best, a layman’s understanding of the situation.”

I wrote to you next on 30 September 2009 and again on17 October 2009 inviting your comments on my online contention that RoSPA, in celebrating the 25th anniversary of the seat belt law, was making ludicrous claims for its effect.

Having failed to elicit any response four years ago now, on the 30th anniversary, I renew my invitation in the form of an open letter. I invite your comments on my recent post 30 years in the jungle with RoSPA .

I invite two comments in particular:

  1. Given that on the third anniversary of the law Lord Nugent, your former president, was claiming that the law was saving 200 lives a year, how do you justify RoSPAs recent anniversary claims of 2400 lives a year – i.e. 60,000 lives saved over 25 years?  (Please note my answer at the time to the evidence on which Lord Nugent relied for his claim of 200 lives saved –     Journal of the Royal Statistical Society Series A, 149, 187-227.)
  2. In the light of the comment by Lord Nugent that RoSPA certainly could not support any road safety measure which discriminated in favour of one section of the community rather than another – do you think that my Organ Harvester Lottery is a fair characterization of the discrimination acknowledged by Allsop et al in their Significance article?

Hoping to hear from you

John

PS  This open letter is not an attempt to settle very old scores. It is an attempt to get RoSPA, an influential organization concerned with risk management, to confront the reality of risk compensation.

When I first became involved in the seat belt debate over 30 years ago the risk compensation hypothesis was jeered at in Parliament: it was bogus,  spurious, eccentric, ludicrous and, more politely, unproven and new. It now merits a Wikipedia entry:

Risk compensation (also Peltzman effect, risk homeostasis) is an observed effect in ethology whereby people tend to adjust their behavior in response to perceived level of risk, behaving less cautiously where they feel more protected and more cautiously where they feel a higher level of risk. The theory emerged out of road safety research after it was observed that many interventions failed to achieve the expected level of benefits but has since found application in many other fields.

That people adjust their behavior in response to changes in perceived levels of risk has now become a commonplace. Where (if) there is debate about the phenomenon it now usually focuses on whether compensation is partial, complete, or more than complete except in the case of seat belts legislation where, for campaigners such as RoSPA, it apparently still does not exist.

Acknowledgement of the phenomenon is essential to an intelligent civilized discussion of any risk management problem. A quick Google search will show that it features in discussions ranging from flood control, anti-lock brakes, cycle helmets, sexual behavior, skydiving, and sub-prime mortgages to shared space and warship safety amongst many others.

RoSPAs continuing repetition of its nonsense claims for the efficacy of Britains seat belt law is in danger of undermining its credibility in other areas where it undertakes useful work.

30 years in the jungle with the Royal Society for the Prevention of Accidents

 31 January marked the 30th anniversary of the implementation of Britains seat belt law. A television interviewer sent to quiz me about my opposition to the law said I reminded him of Hiroo Onoda, the Japanese soldier who spent 30 years in the jungle, fighting on, unaware that the war had been lost. The interview was not broadcast; I assume that when he got back to the office his boss decided that after 30 years the story was no longer newsworthy.

I have actually been in the jungle for more that 30 years. In 1981 I produced a paper entitled The Efficacy of Seat Belt Legislation (published in 1982 by the Society of Automotive Engineers). The paper earned me a gratifying number of column inches in the Hansard report of the 1981 Parliamentary debate that resulted in the passage of the law that was finally implemented in January 1983. The column inches were reasonably equally divided between those who were persuaded by my evidence and those who werent.

The debate was introduced by Conservative MP Ivan Lawrence whose opposition to the law was essentially libertarian. But he welcomed my evidence that the law would not save lives.

Mr. Lawrence This is one of those subjects which stir feelings deeply and genuinely, regardless of party politics. It touches upon freedom, and all of us have been sent to this place to protect, as far as we might, the individual liberties of our constituents against the remorseless hunger of a State and bureaucratic machine trying to gobble it up. ..      we now have some solid evidence that we have never had before. In earlier debates we have heard talk of some figures from Australia that were airily waved about as supporting the compulsion case, but we were never given any statistics from the many other countries experiencing compulsion. Now we have just that. We have a comparative study of road fatality statistics in no fewer than 18 countries, covering about 80 per cent of the world’s car population.

Examining the national statistics which have not been available earlier, Mr. John Adams of University College, London, in January this year published his findings.

Mr. Austin Mitchell (Grimsby) The only one that the hon. and learned Gentleman can dredge up.

Mr. Lawrence It is a darned sight more than anything that has ever been dredged up by the arguers for compulsion on the Opposition Benches, who have never presented this House with a comprehensive analysis of the statistics.

Mr. Mitchell rose

Mr. Lawrence I hope that the hon. Gentleman will be able to catch your eye later, Mr. Speaker. However, I am conscious that many hon. Members wish to speak. Therefore, I hope that the hon. Gentleman will forgive me if I continue.

Mr. Mitchell rose

Mr. Lawrence Mr. John Adams of University College, London, compared Belgium, Denmark, Finland, France, Germany, Holland, Norway, Spain, Switzerland, Sweden, Israel, Australia and New Zealandall countries with a compulsory seat belt law that is enforcedwith Italy, the United States of America, Japan and Great Britain, where there is either no compulsory seat belt law or it is not enforced.

Mr. Adams’ findings are astonishing and unexpected. Although most of the countries experienced a substantial decrease in road accident fatalities in the years following the 1973 oil crisis, the decrease was greater in those countries without a compulsory seat belt law than in those countries with one.

Mr Mitchell rose

Mr. Lawrence The hon. Gentleman will have to carry on

Mr. Deputy Speaker (Mr. Bernard Weatherill) Order. The hon. and learned Gentleman is not giving way.

Mr. Mitchell I wondered whether the hon. and learned Gentleman was giving way.

Mr. Lawrence The hon. Gentleman can carry oil wondering. I should like to complete my case. The promise of massive reductions in the number of deaths because of compulsion simply has not materialised. There is no evidence to show that seat belt compulsiontaking accidents overallsaves lives; nor does it reduce the overall number of injuries. However, the inherent unreliability of comparative statistics makes any positive conclusion about injuries impossible.

That survey has put the cat among the pigeons. Those who have stood up publicly for compulsion have some understandable reluctance to stand on their heads in public, whatever the evidence may be. Others, less publicly committed, will want to stop and reconsider the matter. The first attack made on Mr. Adams’ inconvenient findings was that he had not substantiated his tentative hypothesis that the reason why there seemed to he no reduction in the death rate, and probably an increase after compulsion, was that protecting car occupants from the consequences of bad driving encourages bad driving. As a rock climber may take more risks when wearing a harness, so a car driver may do so. That attack was irrelevant. Mr. Adams was only describing the statistics. He did not set out to explain them and still less to prove an explanation. It is for all of us to consider why those statistics produce that astonishing result. However, it is beyond argument that they produce it.

The next attack came from Lord Nugent, who proposed the amendment in the other place. On 11 Juneonly six weeks agohe said that Mr. Adams’ mistake was to muddle up pedestrians, cyclists, motor cyclists and so on with the only relevant victimsdrivers and front seat passengers. I hear notes of assent and support for that criticism from Opposition Members.

Mr. George Foulkes (South Ayrshire) Nonsense!

Mr. Lawrence I hope that we shall hear some constructive speeches from Opposition Members, rather than “Nonsense!”

Lord Nugent’s criticism of Mr. Adams was patent rubbish since the survey and its appendix contained three graphs, giving information about car occupant fatalities in Sweden and in seven EEC countries. Nevertheless, the criticism was repeated by Dr. Murray MacKay in a letter to The Times on 11 June. Mr. Adams replied, pointing out the error of his critics in The Times in a letter of 16 June. That seems to have made no difference. Once some people have made up their minds, they will not change them, whatever the facts.

On 26 June, the motoring correspondent of The Times repeated the error. Once again, the patient Mr. Adams drew the attention of The Times to the error and once again, the correction was ignored. The leader in The Times stated: Dr. Adams fails to distinguish sufficiently between all road user casualties and those among car occupants.

Mr. Austin Mitchell rose

Mr. Lawrence Mr. Adams has produced 57 graphs. He concludes: Nowhere in all this data can there be found any evidence of the enormous beneficial effect promised by the advocates of legislation. On the contrary the data persistently suggests that the effect of the legislation, if there is one, is perverse. There, for the moment, the evidence rests.

Now, more than 30 years on, the Royal Society for the Prevention of Accidents has issued a press release to celebrate the implementation of the law and to celebrate the role of Lord Nugent (their then president) in its passage. The press release also adds to the patent rubbish of which Ivan Lawrence complained:

31 January 2013

ACTION STILL NEEDED TO SAVE LIVES 30 YEARS AFTER SEATBELT LAW, WARNS RoSPA

Greater compliance with seatbelt legislation could help save lives in road crashes says RoSPA, as it marks the 30th anniversary of the law coming into force.

Well over 60,000 lives have been saved by seatbelts since January 31, 1983, when the Royal Society for the Prevention of Accidents then-President, Lord Nugent of Guildford, won the day for compulsory wearing in the front seat of cars by introducing an amendment to the Transport Bill in the House of Lords. A law making it compulsory to wear seatbelts in the back of cars was introduced in 1991.

RoSPA is shameless. On the 20th anniversary of the law they claimed it had saved 50,000 lives. On the 25th anniversary they claimed 60,000 or 2400 lives a year. Now, on the 30th anniversary, using the scientific method called hand-waving extrapolation they claim well over 60,000.

The nonsense of RoSPAs 60,000 claim was exposed in a number of postings at the time of the 25th anniversary. It was a claim also made in a Department for Transport press release and in a press release by PACTS (the Parliamentary Advisory Council on Transport Safety). The 25th anniversary claims can still be found on the DfT and PACTS websites. PACTS was especially proud of it contribution to the achievement:

On the 31st January 2008, the 25th anniversary of the law change which made front seatbelt wearing compulsory was celebrated. PACTS itself was set up by Barry Sheerman MP as part of the fight to get mandatory seatbelt wearing turned into legislation. Eight years later it became compulsory for all backseat passengers to use seatbelts and it is estimated that since the introduction of the first law change in 1983, seatbelts have prevented 60,000 deaths and over 670,000 serious injuries.

Intriguingly, a few months after this press release four prominent road safety experts who describe themselves as being among the leadership of the Pacts Richard Allsop, Oliver Carsten, Andrew Evans, and Robert Gifford published an article in Significance, a journal of the Royal Statistical Society, in which they advanced a much more modest claim for the number of lives saved by the seat belt law: not 2400 per year but 164.

Also intriguingly, for the first time of which I am aware, serious, statistically-qualified, advocates of seat-belt legislation acknowledged a risk transfer effect:

The clear reduction in death and injury to car occupants is appreciably offset by extra deaths among pedestrians and cyclists.

and

the best estimates from both the analyses summarised here are that extra deaths to vulnerable road users did accompany the introduction of mandatory wearing of seat belts.

Table 1 from Allsop et al summarizes the risk transfer effect for vulnerable road users cyclists, pedestrians and rear sear passengers who were not covered by the seat belt law.

TABLE 1

The seat belt law, they explained, has

two potential unintended effects. First, the affected group may have more collisions, perhaps through taking greater risks, realizing that the likely consequences of collision have been reduced. Second, there may be an increased risk to other road users not protected by the safety measure. In these ways, the reduction in death and injury in the protected group may be partly offset by extra death and injury caused to others.

Dedicated readers of this website will recognize that these two unintended consequences are what for some decades I have been calling the risk compensation effect.

Allsop et al went on to observe that

Whether belts are worn by choice or by law, these unintended effects could lead to the number of deaths to belted car occupants falling by less than the amount one would expect if they were unbelted, and to extra death and injury among pedestrians, cyclists and unbelted occupants. This applies equally to legislating for belt-wearing and to simply encouraging it: it should rule out neither, but be taken into account when considering either.

For 30 years the common response to this argument has been that of David Ennals (above), a supporter of the law, who referred in the parliamentary debate to the extraordinary argument by Dr. John Adams of University College, London that the wearing of seat belts could lead to a false sense of complacency.  He went on to assert that the hypothesis is that ˜protecting an occupant from the consequences of bad driving encourages bad driving, adding there is no evidence of that. He went on to thank RoSPA for producing evidence that proved me wrong yes RoSPA and I go back a long way.

RoSPA never got round to publishing the evidence. Mine passed peer review and was published in the Transactions of the Society of Automotive Engineers as The efficacy of seat belt legislation.

Other comments on my paper made during the debate will convey the flavour

  • that [Adams]piece of research was, as I have said before, bogus.
  • the so-called new evidence of Mr. Adams.
  • Mr. John Adams is a lecturer in geography.
  • He has produced an eccentric paper and has made the preposterous suggestion that the wearing of seat belts encourages people to drive more dangerously.
  • Is it seriously argued that safety devices make people behave more dangerously? I think that the majority of people would consider that a ludicrous proposition.
  • Those of us who have attempted to look at the problem seriously find the evidence in Dr. Adams’s paper highly spurious and bogus.
  • Professor Adams’s thesis is unproven. It is new.

Lord Nugent does indeed deserve much of the credit that RoSPA bestows upon him for the passage of the law. Shortly before the debate he sent a letter on behalf of RoSPA to every MP stating:

Dr Adams has recently published a paper advancing the thesis that wearing seat belts may actually increase road accidents by encouraging a false sense of security. His paper requires detailed analysis with the aid of much background information from the countries concerned before authoritative comment can be made upon it. RoSPA and the Transport and Road Research Laboratory are undertaking these studies, meantime it is relevant to record that Dr Adams does unequivocally state that wearing seatbelts greatly improves the chance of avoiding injury.

The immediate objection to the letter that I made at the time was that it was disingenuous. I had stated that seat belts improve a car occupants chances of avoiding an injury in a crash. The fact that fatalities and injuries, to occupants and others, had not fallen with the implementation of the law was my principal evidence for risk compensation.

A second objection can now be raised. After 30 years, the results of the RoSPA studies commissioned by Lord Nugent have yet to be published, nor has its preposterous contention that the seat belt law is saving 2400 lives a year been justified.

While RoSPAs contribution to the seat belt debate over the last three decades can be dismissed as hand-waving statistical nonsense, the argument of Allsop et al deserves further consideration, if only because its authors have established track records as serious transport safety researchers

Despite having accepted that lives of vulnerable road users have been lost as a result of the seat belt law 268 according to their Table 1 they still argue that seat belt laws should be retained because they save more lives 432 car occupants than they cost. Here is their justification for keeping a law that by their estimation kills 268 vulnerable road users a year

road safety is made up of many individual safety measures and equity among road user  groups across road safety policy as a whole in no way implies that each individual safety measure should in itself benefit all road user groups equitably. Nor does each road safety measure necessarily act only to reduce death and injury: many measures reduce aggregate death and injury by reducing them in incidents of certain kinds by more than they increase them in other kinds. A case in point is installing central barriers on motorways. These prevent many deaths and injuries by preventing errant vehicles on one carriageway crossing onto the other and then having very injurious head-on collisions.

But there are some incidents in which injury or death result from a vehicle striking the barrier and bouncing back onto its own carriageway when the opposing carriageway was fortuitously clear of traffic; had there been no barrier, there would have been no injuries. That is no reason to do away with motorway barriers. The barrier is justified when the deaths and injuries prevented outweigh appreciably the extra deaths and injuries that result from its presence.

It would therefore be a severe constraint on the use of safety measures if any measure were to be ruled out for which the larger number of deaths and injuries saved were differently distributed among road user groups from the smaller number resulting from the measurethe more so because most of us at different times belong to different user groups.

The wearing of seat belts is therefore not exceptional among safety measures in that extra deaths and injuries that may arise from wearing occur in part (only in part because, if wearers drive more riskily, some of the extra deaths and injuries will occur to vehicle occupants) to different road user groups than those among which deaths and injuries are preventedthough the difference is perhaps sharper for belt wearing than for many other measures

Any increase in death and injury to pedestrians and cyclists that stems from wearing of seat belts should be borne in mind in the formulation of road safety policy as a whole and contribute to prioritising safety measures benefitting pedestrians and cyclists. But so long as wearing of belts saves substantially more death and injury among vehicle occupants than the increase among pedestrians and cyclists, the latter does not rule out the provision and wearing of belts. Nor, therefore, does it rule out laws requiring the wearing of belts.

When I first encountered this sophistry (both seat belts and central barriers are measures designed to protect vehicle occupants) justifying the killing of the vulnerable to save the lives of the best protected I put it to one of the authors, Rob Gifford, that the same argument would justify an organ harvester lottery:

In the light of that argument I would welcome your comment on this analogy: an Organ Harvester Lottery in which healthy people will be selected at random (not unlike road accidents) to donate organs to people in need of them. One heart and one liver could save two lives at the cost of one slightly better than the ratio that you argue justifies keeping the seat belt law. Throw in two kidneys and you get a four to one ratio of lives saved to lives lost. This is a gruesome but not wholly inappropriate analogy given the myth circulating at the time (and still) that the success of the seat law was responsible for increasing the shortage of donor organs.

I have yet to receive a challenge from the PACTS authors to the appropriateness of this analogy.

Let us conclude with an examination of the appropriateness of the analogy comparing me to Hiroo Onoda with which we began. According to this analogy my war against seat belt laws was lost over thirty years ago with the passage of Britains first seat belt law.

In one sense the war has indeed been lost. The myth of the efficacy of seat belts and seat belt laws is now so deeply entrenched that it appears to be immune to evidence. It is routinely advanced in support of other safety regulation, such as compulsory cycle helmets, as an example of the efficacy of measures that compel people to be safer than they voluntarily choose to be.

An acceptance of the myth leads to its routine insertion into all sorts of other arguments. I offer a recent example from the New York Times an article by a Cambridge philosopher, Huw Price, explaining why he was working to establish in Cambridge the Centre for the Study of Existential Risk (C.S.E.R.).

His project he likens to a societal seat belt. He elaborates the analogy:

our fate is in the hands, if thats the word, of what might charitably be called a very large and poorly organized committee collectively short-sighted, if not actually reckless, but responsible for guiding our fast-moving vehicle through some hazardous and yet completely unfamiliar terrain.

and

Second, remember that all the children all of them are in the back. We thrill-seeking grandparents may have little to lose, but shouldnt we be encouraging the kids to buckle up?

A fact not widely known to those who might be persuaded by this analogy is that, when Britain passed a law in 1989 requiring children in back seats to belt up, the number of children killed in back seats increased (see Risk, pp 126-127). My provisional explanation, pending a more convincing one: when the children were secure in their government approved safety harnesses, parents reverted to their customary, vigorous, style of braking, cornering, and accelerating.

So common is the deployment of the efficacious-seat-belt analogy that I, for peace of mind, have learned, to let it pass.  But in this case, sensitised by the impending 30th anniversary celebrations, I ventured a comment.

The myth of the efficacy of seat belt legislation

Why should the belief in the efficacy of the law have persisted for so long in the face of so much evidence that it hasnt worked? Myth has been called  “ideology in narrative form”.

The narrative. Before the passage of the law in 1981 there had been numerous parliamentary debates about a seat belt law. Over this period the evidence, and popular support, appeared to grow stronger. Experiments with crash test dummies provided convincing graphic evidence of the protection afforded by seat belts in crashes. And in the early 1970s all the Australian states passed seat belt laws that appeared to arrest a long established rising trend of road accident deaths. The belief in the potential road safety legislation to save lives gained momentum.

By the time of the 1981 debate the law had acquired a large number of influential champions practically any organisation with an interest in road safety including the British Medical Association, the Royal society for the Prevention of Accidents, the Royal College of Surgeons, the Royal College of Nursing, the Society of Automotive Manufacturers and Traders, and the Automobile Association. Lord Avebury offered this list in a House of Lords debate as compelling evidence for legislation: Why, he asked, would all these institutions seek to mislead the public?

But none of these organisations, and none of the countries that followed the lead of Australia, produced any compelling evidence that legislation had saved lives. They all cited the original, misconstrued, Australian evidence, or other people citing the Australian evidence, or other people citing other people citing the Australian evidence.

Ever since, the champions of the law have felt free to assert its effectiveness with impunity. Five years ago I pointed out to the Department for Transport, the Parliamentary Council for Transport Safety and the Royal Society for the Prevention of Accidents that their 25th Anniversary claim that the law had saved 60,000 lives was preposterous. But the claim can still be found on their websites. Echoing Lord Averbury one might well ask, why would these institutions seek to mislead the public?

 

 

 

 

Is ISO 31000 fit for purpose?

The debate

Is ISO 31000 fit for purpose is the headline above a debate published in the June edition of Risk Management Professional for online version click here. The debate consisted of an abbreviated version of my blog ISO 31 000: Dr Rorschach meets Humpty Dumpty -­ and a rebuttal by Grant Purdy, one of the principal authors of the ISO Standard.

Debate and rebuttal have been enclosed in quotation marks because a serious debate did not take place. The rebuttal is entitled Never perfect, but inclusionary, practical and widely accepted, and consists mostly of an explanation of (and excuse for) any imperfections that it might have.

It observes that:

  • standards may not reflect the ‘best available’ practices and leadingthinking.
  • a standard can be biased because of prevailing influences in the committeethat prepared it.
  • significant compromises are often required to obtain consensus in acommittee. and concedes that

¢ it would be naïve to think that ISO 31000 could not suffer from any of the problems described above.

The rebuttal further acknowledges some fudging and some unnecessary complexity. None of these admissions of possible imperfection are related to any specific parts of ISO 31000; readers are left to work out for themselves where within the document they might be found.

Is it fit?

But lets move on to the purpose of ISO 31000. … read the full essay here  

ISO 31000: the debate warms up

Until recently most online discussion of ISO 31000 has been confined to a friendly Linkedin site for supporters: two quotations – I know the ISO 31000 and think it’s almost perfect and I think the ISO 31000 definition of risk is great will convey the flavor of the critical discussion to be found on the site.

The discussion is frequently ad hominem in a nice way. Participants who agree with each other declare their mutual respect warmly I value your connection/friendship dearly. But when skeptics or heretics attempt to join the discussion it can turn ad hominem in a nasty way.

Indeed outsiders do not even have to attempt to join the Linkedin forum; they can be dragged in. This happened recently to Robert Kaplan of the Harvard Business School. After speaking at a conference he was approached by one of the audience and asked his opinion about ISO 31000: When I personally asked him afterwards to clarify how he saw ISO 31000 fitting in, he put it in the same basket as any other standard and ˜not relevant (implying not relevant to the future advancement of ERM).

When asked, he said something that implied something. This was sufficient to involve him (in absentia} in a heated discussion on the ISO Linkedin site that degenerated into personal abuse.

One participant described him as another arrogant potion peddling merchant that really should be treated as such although, this one seems to hide behind the curtain and title of a university doctrine [sic] to make the whole experience feel authentic for the next manager that is about to be ripped off.

The convenor of the discussion declared at one point, it is really upsetting that Dr. Kaplan’s team’s research is limited [sic] with LinkedIn ISO 31000 group. They did not asked [sic] permission for that and the legal rights are belongs [sic] to us not to him. We need to sue him for this. It is not clear what this means or what that is. But we need to sue him!

The tone is consistently ad hominem, varying between nasty and nice depending on the participants perspective: these people are a liability for the progression of risk management, excellent comments, I have no desire to waste my time with charlatans, 100% agree etc. etc.

At one point in the discussion the ubiquitous Arnold Schanfield of Manhattanville College denounces Peter Bonisch of Paradigm Risk as dishonourable. He elaborates We [i.e. the ISO fan club] are all offended by your remarks most of which cannot be supported and defended and your intentions are indeed dishonorable. If they were honorable you would be taking part in a discussion on what we need to do to move the discipline forward. Instead you have chosen to attack those individuals/professionals with very strong risk management backgrounds in a non stop barrage of $50,000 words [sic]. As with much else on this Linkedin site it is not entirely clear what this is intended to mean (or how the price of words was calculated) other than that Schanfield stands by his view that Bonisch is dishonourable.

Bonisch has now retreated to the calmer, more dignified and coherent environment of his own blog and published an essay worth reading.

He begins by quoting a statement by Kaplan that I find difficult to argue with. It suggests that his (Kaplans) position is more nuanced than that attributed to him following an oral conversation:

Standards and innovation have an inherent tension between each other, in some cases they can be mortal enemies. We standardize when we understand a process very well and want to ensure that everyone follows the same processes and measurements because they have been proven to yield superior results. But in an environment with limited knowledge and experience, premature standard setting will inhibit innovation, exploration and learning. We can standardize around preventable risks because managers do understand them well, and have developed excellent processes to prevent them from occurring. But we are just learning about the management of strategy risks and external, non-preventable risks. To think we can standardize the best practices for managing these two risk categories through an ISO-based process seems like a highly risky proposition for risk professionals to be engaged in with our present body of knowledge.

Skipping to the end, Bonisch concludes, tellingly, by calling attention to an ISO 31000 injunction that I touched upon in my last blog on this subject: the risk manager should generate a comprehensive list of risks. ISO 31000 leaves the reader in no doubt about the importance of comprehensiveness:
ï‚· Managements framework should be comprehensive
ï‚· Comprehensive identification is critical
ï‚· The organization should have a comprehensive understanding of its risks
ï‚· Enhanced risk management includes comprehensive, fully defined and fully accepted accountability for risks, controls and risk treatment tasks
ï‚· Enhanced risk management includes comprehensive and frequent reporting of risk management performance
ï‚· There is a need for risk treatment against properly established and comprehensive risk criteria and
ï‚· Comprehensive and frequent external and internal reporting contributes substantially to effective governance within an organization.

In the face of irreducible uncertainty, Bonisch asks, how can analysis of its effects ever be comprehensive?

Consider the comprehensive challenge through the eyes of a project manager tasked with producing a comprehensive event tree. It would be a strange tree, unknown to nature. It would have an infinite number of branches of varying lengths and numbers of branching points.

The branching points on some branches, lets call these branches probabilistically-known knowns, can be assigned probabilities with relatively small error bands. This is the realm of someone, such as an actuary in a motor insurance company, working with large and stable actuarial databases.

Other branches, lets call them known unknowns because the branching points represent imaginable possibilities, and the probabilities assigned to them are speculative; they have such wide error bands that they are effectively quantified expressions of ignorance. The fire that closed the Channel Tunnel for six months shortly after it opened had been assigned a probability in an event tree of somewhere between zero and 2.8 per billion journeys.

But the largest number of branches and branching points, a number that renders my contention of infinite unchallengeable, is the number of the unknown unknowns. We now live in a world inhabited by more than seven billion risk managers interacting reflexively with each other, and with natural hazards. The computer required to manage such a system is unimaginable, because once it started managing, it too would get caught up in myriad infinitely reflexive loops.

If one wants certainty one must turn to the fault tree. The event tree is a map of infinite possible futures. The engineers fault tree is a map of the post-accident past, and it commonly has only one branch called culpable negligence. (see dangerous trees)

So, what hope of demonstrating the (in)effectiveness of ISO 31000 or COSO or any other risk management standard? Concorde, before it crashed, could claim to be the safest airliner in the world zero fatalities. After it crashed it became the airliner with the highest fatality rate per mile flown because it had flown so few miles.

Good luck to those who hope to demonstrate the superiority of their risk management standards. Enron and Concorde had lots of fans until they crashed comprehensively. The ISO 31000 standard says that the risk manager should allocate appropriate resources for risk management. What it does not say is how, with finite resources and infinite uncertainty, one should define appropriate.

Post hoc, trees are dangerous

Last night (11/4/2012) I took part in an interesting Radio 3 discussion programme called Night Waves . The first contributor, Jonathan Haidt, was fascinating and Ive just ordered his new book The Righteous Mind.

In the discussion he made a point that resonated with a problem that I have been wrestling with: discussing the relationship between reason and emotion he said, reason is mostly post hoc. Risk management decisions are taken pre hoc. They are decisions taken with insufficient information. They are decisions about an uncertain future. Post hoc, after the event/accident, there is usually much more information to which one can apply cause and effect reasoning. Before the event the links between causes and effects are either unknown or shrouded in estimates of probability, with large error bands attached. The information gap confronting a risk manager is readily filled by emotion – the natural instinctive state of mind deriving from ones circumstances, mood, or relationships with others.

Event Trees, beloved of engineers and project managers as a method for describing the risks associated with big projects, represent a denial of this reality (see Dangerous Trees?). All of the branching points on each limb of their trees must be assigned probabilities, usually with large error bands. By the time one reaches the tip ends of the branches the compound error bands are too large to fit on the page, so the magnitude of the uncertainty is ignored.

In the dangerous trees example just referred to an actual tree blew down and killed some one. In Britain, averaged over the whole country, the statistical risk of such a thing happening in a given year is about 1 in 10 million. Britains Health and Safety Executive describes risks of 1 in 1 million as adequately controlled. But after the event different sorts of reasoning are brought to bear. The police, who are predisposed to allocate blame in such circumstances, arrested the head estate warden and the estate manager on suspicion of negligent manslaughter. The coroner declared, a year and a half later, that all mature trees on the estates land adjacent to paths should be inspected and logged a requirement that, if applied to all the National Trusts properties (whose estate it was) might have bankrupted the institution.

Post hoc Event Trees are often transformed into Fault Trees that, with the aid of clear facts and reason, are transformed into trees with only one branch called culpable negligence. What pre hoc appeared a risk of 1 in 10 million, post hoc becomes a risk of 1 in 1.

After a year and a half the Crown Prosecution Service, dropped the case for lack of evidence, but a threat of dire consequence still hangs over the National Trust. Given the many millions of mature trees in Britain, the risk assessment and risk management implications of the coroners judgment, pursued to their logical conclusion would require the diversion of enormous resources to inspection and/or the execution of countless mature trees that could not be guaranteed 100 percent safe and who would offer that assurance in our litigious times? This experience is now colouring the (emotional?) judgment of Britains tree managers. In deciding what trees must be executed the legal risk now far outweighs the physical risk.