![]() ![]() Although risk matrices can indeed be very useful if probability and consequence values are positively correlated, they can be worse than useless when probability and consequence values are negatively correlated. “.the common assumption that risk matrices, although imprecise, do some good in helping to focus attention on the most serious problems and in screening out less serious problems is not necessarily justified. Additionally, the words used to describe probability (compare these two definitions of improbable: “virtually improbable and unrealistic” with “ would require a rare combination of factors to cause an incident”) lead to inconsistencies.īefore summing up, I give you an interesting extract from Cox’s paper: A study by Budescu et al (2009) showed that “very likely” was assigned to probabilities varying from 0.43 to 0.99 because context and personal attitude to risk will always influence a person’s perception of consequence. Thomas, Reidar and Bratvold talk about centring bias, a phenomenon in which 75% of the probability ratings assessed were centred around 2,3 and 4, thereby effectively reducing the matrix to a 3x3! Also, probability ratings are often necessarily ambiguous and open to interpretation. ![]() For example, the risk of one or two deaths would have to be categorised alongside total hull loss resulting in mass casualties, alround which feel wrong to me. The boundary between hazardous and catastrophic is blurred because there is nowhere to categorise a single, or a “few” deaths. Consider ICAO’s “Example safety risk severity table” (table 1). For me, he has named one of the most significant errors “range compression” which can lead to risks with significantly different values being placed in the same cell. The numbers are drawing incorrect comparisons between risks, suggesting that Remote/hazardous is twice as risky as improbable/major, as a result of committing the mathematical no-no of multiplying an ordinal scale.Ĭox has written extensively about the use of risk matrices and investigates how the use of ordinal scales can lead to errors in decision making. Multiplication cannot be applied to an ordinal scale, but this is what appears to have been done in the CAA UK’s CAP 795, Safety Management Systems (SMS) guidance for organisations. Like 70 degrees is not twice as hot as 35 degrees. Using the previous example, it’s obvious that “excruciating” is not twice as painful as “some”. The numbers are irrelevant as long as the order stays the same. Ordinal responses may be transformed in any way that preserves their order, which in a 5x5 risk matrix could be 1-5 or 0,5,37,40 and 103. What’s excruciating to me maybe merely “a little painful” to you. ![]() This also emphasises the subjective nature of the scale. The responses go from least to most pain, but it’s not clear whether the difference between “none” and “a little” is bigger, smaller, or the same as the difference between “a lot” and “excruciating”. Questions with subjective responses are often ordinal, for example, “how much pain are you in?” could be answered with “none”, “a little”, “some”, “a lot”, “excruciating”. An ordinal scale has no fixed distance between the levels the numbers represent a rank position. From the numerous and varied scales I have encountered in aviation and elsewhere, the common factor is they are typically ordinal scales. My research highlighted that there is no scientific method of designing the scale used in a risk matrix. I have always been mildly sceptical of risk matrices, struggling to see how all that risk data could be condensed and simplified into a single box, and still remain meaningful. A risk matrix is meant to help us categorise, prioritise and compare risks, so what difference does 5 or 4 rows or columns really make? I thought it would depend on how precisely the severity and likelihood ranges have been defined but what I found out was so much more. I was asked the other day what advantage a 5x5 risk matrix has over a 4x5 and it started me thinking. I am posting it again because I am in the process of writing some thoughts on risk classification and it links nicely to this. I liked that comments talked about "disrupting" safety (thank you Horst Simon) and that we should always be looking forward. It was generally agreed that Risk Management activities such as mapping out possible accident scenarios, in identifying the critical controls and ensuring they are adequate and reliable are crucial, and risk matrices are just one way, among many, of presenting the result of your Risk analysis. ![]() I posted this article a while ago and was overwhelmed by the response and discussion it generated. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |