@norootcause@hachyderm.io
@norootcause@hachyderm.io avatar

norootcause

@norootcause@hachyderm.io

Student of complex systems failures, resilience engineering, cognitive systems engineering. Will talk your ear off about learning from incidents in software.

This profile is from a federated server and may be incomplete. Browse more on the original instance.

norootcause, to random
@norootcause@hachyderm.io avatar

New blog post: https://surfingcomplexity.blog/2024/03/16/when-theres-no-gemba-to-go-to/

norootcause, to random
@norootcause@hachyderm.io avatar

Doing some Toyota-related reading and thread-following-Googling brought me to this page: https://www.lean.org/lexicon-terms/gemba/

In the table on that page, one of these things is not like the others.

norootcause,
@norootcause@hachyderm.io avatar

Code is an artifact generated by the work, it’s not where the work gets done.

norootcause, to random
@norootcause@hachyderm.io avatar

If you don’t know how the work actually gets done, your proposed improvements are unlikely to have the effects that you expect.

norootcause, to random
@norootcause@hachyderm.io avatar

I don’t know what the programming language of the future will look like, but I know it will be called Java.

norootcause, to random
@norootcause@hachyderm.io avatar

Brb, typing “now make it even more reliable” into the LLM chat interface.

norootcause, to random
@norootcause@hachyderm.io avatar

You can’t solve coordination problems with accountability.

norootcause, to random
@norootcause@hachyderm.io avatar

One of the few things that becomes less clear with hindsight is the uncertainty that people faced in the moment when dealing with an incident.

norootcause, to random
@norootcause@hachyderm.io avatar

In a complex system, there isn’t a “safety” knob that you can just turn to the right to increase safety. Safety features increase complexity (new failure modes!) and have opportunity costs (finite resources!). Every intervention involves a tradeoff.

norootcause, to random
@norootcause@hachyderm.io avatar

One day I’d like to write a “Senior Software Engineer” chatbot that randomly responds to questions with answers like:

  • “It depends…”
  • “What are you trying to accomplish?”
  • “You’re asking the wrong question.”
norootcause, to random
@norootcause@hachyderm.io avatar

“Practitioners can only act on the knowledge they have.” –
D.D. Woods et al., Behind Human Error

norootcause,
@norootcause@hachyderm.io avatar

“Devices that are internally complex but superficially simple encourage practitioners to adopt overly simplistic models of device operation and to develop high confidence that these models are accurate and reliable.”

norootcause, to random
@norootcause@hachyderm.io avatar

Twasn't actually ever thus

norootcause, to random
@norootcause@hachyderm.io avatar

Two of the biggest sources of incidents I’ve seen are:

  1. Legacy code
  2. Migrating away from legacy code

The conclusion is clear: you should only write non-legacy code

norootcause,
@norootcause@hachyderm.io avatar

@stuartmarks Right??? We should just stop writing bugs altogether.

norootcause,
@norootcause@hachyderm.io avatar

@jawnsy @stuartmarks The real root cause of incidents is our inability to see into the future.

norootcause, to random
@norootcause@hachyderm.io avatar

Distributed systems are spooky-failure-modes-at-a-distance

norootcause, to random
@norootcause@hachyderm.io avatar

Moving-the-needle-as-a-service

recursive, (edited ) to random
@recursive@hachyderm.io avatar

Technical words that sound worse to people outside of that particular field:

degenerate (e.g. https://en.wikipedia.org/wiki/Degeneracy#Science,_mathematics,_and_medicine)
regime (e.g. https://en.wikipedia.org/wiki/Regime_shift)

I'm trying to think of others. Please no outright slurs, we can let those die.

(Edit: folks on Discord mentioned:
"reaping zombie children" (Unix process management)
"performed an illegal operation" (computers in general)
"exploit" (compare resource usage vs. computer security)
)

norootcause,
@norootcause@hachyderm.io avatar

@recursive It's an older story, and it's the name of a software package, but, "subversion": https://www.npr.org/sections/money/2009/07/whats_the_deal_with_that_goldm.html

norootcause, to random
@norootcause@hachyderm.io avatar

Claim: philosophy is even more important to engineering than to science because the stakes are higher

norootcause,
@norootcause@hachyderm.io avatar

@kellogh Philosophy. Ethics is a subset.

norootcause, to random
@norootcause@hachyderm.io avatar

Continually doing work to keep the system up: https://willempennings.nl/balancing-cube/

norootcause, to random
@norootcause@hachyderm.io avatar

New blog post (book review): https://surfingcomplexity.blog/2024/02/11/book-review-trust-in-numbers/

thomasfuchs, to random
@thomasfuchs@hachyderm.io avatar

They: referring to “the big game”

Me: automatically thinking of

norootcause,
@norootcause@hachyderm.io avatar

@thomasfuchs How did they ever get Mark Hamill, Malcolm McDowell and John Rhys-Davies to star in this game???

norootcause, to random
@norootcause@hachyderm.io avatar

In hindsight, foresight is pretty tough to pull off: https://jabberwocking.com/investors-have-no-idea-what-theyre-doing/

norootcause,
@norootcause@hachyderm.io avatar

@marick @twasink On the topic of central banks, I thought this was pretty funny: https://x.com/CarlHedgren/status/1755697181091099068?s=20

(Is it gauche to post twitter links here?)

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • JUstTest
  • tacticalgear
  • thenastyranch
  • ngwrru68w68
  • magazineikmin
  • khanakhh
  • rosin
  • mdbf
  • Youngstown
  • slotface
  • everett
  • cisconetworking
  • kavyap
  • DreamBathrooms
  • anitta
  • InstantRegret
  • Durango
  • osvaldo12
  • ethstaker
  • modclub
  • GTA5RPClips
  • Leos
  • cubers
  • tester
  • normalnudes
  • megavids
  • provamag3
  • lostlight
  • All magazines
    •