I see it as the continuation of a very old problem. Old school engineering didn’t have any standards until a bunch of people died over and over and the public demanded change. The railroads, construction tycoons, factory owners, mine operators etc all bitterly fought, and still fight, engineering safety requirements. Computer industries have continued this. They all oppose public action, hide negative information, and try to pin blame for conspicuous failures on individuals rather than systemic rot.
I think also because of the relatively less visceral nature of software catastrophes we don’t have a culture of safety. That’s not to say software errors can’t cause horrific accidents but the power grid going down and causing a dozen people in the service area to die is less traumatic than a bridge collapsing and sending a dozen people into an icy river. That’s an extreme example but my point is that humans undervalue harms that are seen as less acutely, physically brutal and software just seems more abstract.
Most of us aren’t working on power grid either, so when you start trying to quantify our software’s risks you have to speak to “harms” rather than just crimes like negligence, and then you expose this huge contradiction about how responsibility is allocated socially. Like, not only should engineers, pilots, and doctors have higher responsibility to prevent harm, but so should cops, journalists, politicians, billionaires, etc.
So the risks are undervalued and both intentionally and unconsciously minimized. The result is most of us who’ve seen the inside are quietly horrified and that’s the end of it.
I don’t know what the answer is except unignorable tragedies because that seems to be the only thing powerful enough to build regulations which are constantly being eroded.
… my point is that humans undervalue harms that are seen as less acutely, physically brutal …
i think you’ve excellently summed up the reason why horrible things like a genocide are allowed happen; so long as we’re not allowed to witness the atrocities in the media and we dehumanize them, we will never understand the needful visceral reaction to stop it.
So the risks are undervalued and both intentionally and unconsciously minimized. The result is most of us who’ve seen the inside are quietly horrified and that’s the end of it.
i wonder if something like the Hippocratic Oath could help; but then again the same physicians to took the oath committed atrocities & human rights violations under the eugenics programs/pushes that that the united states suffered from in the early to mid 20th century.
I see it as the continuation of a very old problem. Old school engineering didn’t have any standards until a bunch of people died over and over and the public demanded change. The railroads, construction tycoons, factory owners, mine operators etc all bitterly fought, and still fight, engineering safety requirements. Computer industries have continued this. They all oppose public action, hide negative information, and try to pin blame for conspicuous failures on individuals rather than systemic rot.
I think also because of the relatively less visceral nature of software catastrophes we don’t have a culture of safety. That’s not to say software errors can’t cause horrific accidents but the power grid going down and causing a dozen people in the service area to die is less traumatic than a bridge collapsing and sending a dozen people into an icy river. That’s an extreme example but my point is that humans undervalue harms that are seen as less acutely, physically brutal and software just seems more abstract.
Most of us aren’t working on power grid either, so when you start trying to quantify our software’s risks you have to speak to “harms” rather than just crimes like negligence, and then you expose this huge contradiction about how responsibility is allocated socially. Like, not only should engineers, pilots, and doctors have higher responsibility to prevent harm, but so should cops, journalists, politicians, billionaires, etc.
So the risks are undervalued and both intentionally and unconsciously minimized. The result is most of us who’ve seen the inside are quietly horrified and that’s the end of it.
I don’t know what the answer is except unignorable tragedies because that seems to be the only thing powerful enough to build regulations which are constantly being eroded.
i think you’ve excellently summed up the reason why horrible things like a genocide are allowed happen; so long as we’re not allowed to witness the atrocities in the media and we dehumanize them, we will never understand the needful visceral reaction to stop it.
i wonder if something like the Hippocratic Oath could help; but then again the same physicians to took the oath committed atrocities & human rights violations under the eugenics programs/pushes that that the united states suffered from in the early to mid 20th century.