Normalization of Deviance

This came up at work: http://danluu.com/wat/ – The idea being that, over time, we come to accept that "WTFs" of a job as being normal, and we accept deviance, both from what is reasonable and what is policy, as being routine and acceptable.

That article is in the context of the tech industry, but references and builds upon a 2011 article, The normalization of deviance in healthcare delivery. It makes the argument that a lot of catastrophes require a confluence of multiple things going wrong at once, but many of them are minor on their own. In one example, an anesthetist essentially killed a patient by forgetting to turn their oxygen back on, which was the "big" cause–but as a contributing factor, they had also silenced the patient monitor that was there specifically to catch stuff like this. Silencing a patient monitor probably ought not to happen, but it's not normally a big deal–except when it is. In this case, it would have saved a life.

Both are quite interesting reads. The medical one also references the Bhopal disaster, which is an even more absurd example of where a ton of seemingly-minor-on-their-own failures contributed to killing thousands and injuring more than a half-million:

This reminds me of another paper I'd read, which also looks to be a medical paper easily adapted to the tech industry: How Complex Systems Fail (PDF). One particularly salient excerpt:

Overt catastrophic failure occurs when small, apparently innocuous failures join to create opportunity for a systemic accident. Each of these small failures is necessary to cause catastrophe but only the combination is sufficient to permit failure. Put another way, there are many more failure opportunities than overt system accidents.
Show Comments